{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Collaboration and Competition\n",
    "\n",
    "---\n",
    "\n",
    "In this notebook, you will learn how to use the Unity ML-Agents environment for the third project of the [Deep Reinforcement Learning Nanodegree](https://www.udacity.com/course/deep-reinforcement-learning-nanodegree--nd893) program.\n",
    "\n",
    "### 1. Start the Environment\n",
    "\n",
    "We begin by importing the necessary packages.  If the code cell below returns an error, please revisit the project instructions to double-check that you have installed [Unity ML-Agents](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Installation.md) and [NumPy](http://www.numpy.org/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import json\n",
    "\n",
    "import numpy as np\n",
    "import random\n",
    "import copy\n",
    "from collections import namedtuple, deque\n",
    "\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will start the environment!  **_Before running the code cell below_**, change the `file_name` parameter to match the location of the Unity environment that you downloaded.\n",
    "\n",
    "- **Mac**: `\"path/to/Tennis.app\"`\n",
    "- **Windows** (x86): `\"path/to/Tennis_Windows_x86/Tennis.exe\"`\n",
    "- **Windows** (x86_64): `\"path/to/Tennis_Windows_x86_64/Tennis.exe\"`\n",
    "- **Linux** (x86): `\"path/to/Tennis_Linux/Tennis.x86\"`\n",
    "- **Linux** (x86_64): `\"path/to/Tennis_Linux/Tennis.x86_64\"`\n",
    "- **Linux** (x86, headless): `\"path/to/Tennis_Linux_NoVis/Tennis.x86\"`\n",
    "- **Linux** (x86_64, headless): `\"path/to/Tennis_Linux_NoVis/Tennis.x86_64\"`\n",
    "\n",
    "For instance, if you are using a Mac, then you downloaded `Tennis.app`.  If this file is in the same folder as the notebook, then the line below should appear as follows:\n",
    "```\n",
    "env = UnityEnvironment(file_name=\"Tennis.app\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:unityagents:\n",
      "'Academy' started successfully!\n",
      "Unity Academy name: Academy\n",
      "        Number of Brains: 1\n",
      "        Number of External Brains : 1\n",
      "        Lesson number : 0\n",
      "        Reset Parameters :\n",
      "\t\t\n",
      "Unity brain name: TennisBrain\n",
      "        Number of Visual Observations (per agent): 0\n",
      "        Vector Observation space type: continuous\n",
      "        Vector Observation space size (per agent): 8\n",
      "        Number of stacked Vector Observation: 3\n",
      "        Vector Action space type: continuous\n",
      "        Vector Action space size (per agent): 2\n",
      "        Vector Action descriptions: , \n"
     ]
    }
   ],
   "source": [
    "# For Local Machine\n",
    "from unityagents import UnityEnvironment\n",
    "env = UnityEnvironment(file_name=\"./Tennis_Linux/Tennis.x86_64\")\n",
    "\n",
    "# # For Udacity Workspace\n",
    "# !pip -q install ./python\n",
    "# from unityagents import UnityEnvironment\n",
    "# env = UnityEnvironment(file_name=\"/data/Tennis_Linux_NoVis/Tennis\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Environments contain **_brains_** which are responsible for deciding the actions of their associated agents. Here we check for the first brain available, and set it as the default brain we will be controlling from Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the default brain\n",
    "brain_name = env.brain_names[0]\n",
    "brain = env.brains[brain_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Examine the State and Action Spaces\n",
    "\n",
    "In this environment, two agents control rackets to bounce a ball over a net. If an agent hits the ball over the net, it receives a reward of +0.1.  If an agent lets a ball hit the ground or hits the ball out of bounds, it receives a reward of -0.01.  Thus, the goal of each agent is to keep the ball in play.\n",
    "\n",
    "The observation space consists of 8 variables corresponding to the position and velocity of the ball and racket. Two continuous actions are available, corresponding to movement toward (or away from) the net, and jumping. \n",
    "\n",
    "Run the code cell below to print some information about the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of agents: 2\n",
      "Size of each action: 2\n",
      "There are 2 agents. Each observes a state with length: 24\n",
      "The state for the first agent looks like: [ 0.          0.          0.          0.          0.          0.\n",
      "  0.          0.          0.          0.          0.          0.\n",
      "  0.          0.          0.          0.         -6.65278625 -1.5\n",
      " -0.          0.          6.83172083  6.         -0.          0.        ]\n"
     ]
    }
   ],
   "source": [
    "# reset the environment\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "\n",
    "# number of agents \n",
    "num_agents = len(env_info.agents)\n",
    "print('Number of agents:', num_agents)\n",
    "\n",
    "# size of each action\n",
    "action_size = brain.vector_action_space_size\n",
    "print('Size of each action:', action_size)\n",
    "\n",
    "# examine the state space \n",
    "states = env_info.vector_observations\n",
    "state_size = states.shape[1]\n",
    "print('There are {} agents. Each observes a state with length: {}'.format(states.shape[0], state_size))\n",
    "print('The state for the first agent looks like:', states[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Take Random Actions in the Environment\n",
    "\n",
    "In the next code cell, you will learn how to use the Python API to control the agents and receive feedback from the environment.\n",
    "\n",
    "Once this cell is executed, you will watch the agents' performance, if they select actions at random with each time step.  A window should pop up that allows you to observe the agents.\n",
    "\n",
    "Of course, as part of the project, you'll have to change the code so that the agents are able to use their experiences to gradually choose better actions when interacting with the environment!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# for i in range(1, 10):                                      # play game for 5 episodes\n",
    "#     env_info = env.reset(train_mode=False)[brain_name]     # reset the environment    \n",
    "#     states = env_info.vector_observations                  # get the current state (for each agent)\n",
    "#     scores = np.zeros(num_agents)                          # initialize the score (for each agent)\n",
    "#     while True:\n",
    "#         actions = np.random.randn(num_agents, action_size) # select an action (for each agent)\n",
    "#         actions = np.clip(actions, -1, 1)                  # all actions between -1 and 1\n",
    "#         env_info = env.step(actions)[brain_name]           # send all actions to tne environment\n",
    "#         next_states = env_info.vector_observations         # get next state (for each agent)\n",
    "#         rewards = env_info.rewards                         # get reward (for each agent)\n",
    "#         dones = env_info.local_done                        # see if episode finished\n",
    "#         scores += env_info.rewards                         # update the score (for each agent)\n",
    "#         states = next_states                               # roll over states to next time step\n",
    "#         print(actions)\n",
    "#         print(rewards)\n",
    "#         print(dones)\n",
    "#         if np.any(dones):                                  # exit loop if episode finished\n",
    "#             break\n",
    "#     print('Score (max over agents) from episode {}: {}'.format(i, np.max(scores)))\n",
    "# #     print(rewards)\n",
    "# #     print(dones)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When finished, you can close the environment."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. It's Your Turn!\n",
    "\n",
    "Now it's your turn to train your own agent to solve the environment!  When training the environment, set `train_mode=True`, so that the line for resetting the environment looks like the following:\n",
    "```python\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "#MADDPG\n",
    "def seeding(seed=10):\n",
    "    random.seed(seed)\n",
    "    np.random.seed(seed)\n",
    "    torch.manual_seed(seed)\n",
    "\n",
    "def hidden_init(layer):\n",
    "    fan_in = layer.weight.data.size()[0]\n",
    "    lim = 1. / np.sqrt(fan_in)\n",
    "    return (-lim, lim)\n",
    "\n",
    "\n",
    "class Actor(nn.Module):\n",
    "    \"\"\"Actor (Policy) Model.\"\"\"\n",
    "\n",
    "    def __init__(self, state_size, action_size, fc1_units=256, fc2_units=128):\n",
    "        \"\"\"Initialize parameters and build model.\n",
    "        Params\n",
    "        ======\n",
    "            state_size (int): Dimension of each state\n",
    "            action_size (int): Dimension of each action\n",
    "            fc1_units (int): Number of nodes in first hidden layer\n",
    "            fc2_units (int): Number of nodes in second hidden layer\n",
    "        \"\"\"\n",
    "        super(Actor, self).__init__()\n",
    "        self.fc1 = nn.Linear(state_size, fc1_units)\n",
    "        self.fc2 = nn.Linear(fc1_units, fc2_units)\n",
    "        self.fc3 = nn.Linear(fc2_units, action_size)\n",
    "        self.reset_parameters()\n",
    "\n",
    "    def reset_parameters(self):\n",
    "        self.fc1.weight.data.uniform_(*hidden_init(self.fc1))\n",
    "        self.fc2.weight.data.uniform_(*hidden_init(self.fc2))\n",
    "        self.fc3.weight.data.uniform_(-3e-3, 3e-3)\n",
    "\n",
    "    def forward(self, state):\n",
    "        \"\"\"Build an actor (policy) network that maps states -> actions.\"\"\"\n",
    "        x = F.elu(self.fc1(state))\n",
    "        x = F.elu(self.fc2(x))\n",
    "        return F.tanh(self.fc3(x))\n",
    "\n",
    "\n",
    "class Critic(nn.Module):\n",
    "    \"\"\"Critic (Value) Model.\"\"\"\n",
    "\n",
    "    def __init__(self, full_state_size, full_action_size, fcs1_units=256, fc2_units=128):\n",
    "        \"\"\"Initialize parameters and build model.\n",
    "        Params\n",
    "        ======\n",
    "            state_size (int): Dimension of both agents states\n",
    "            action_size (int): Dimension of both agents actions\n",
    "            fcs1_units (int): Number of nodes in the first hidden layer\n",
    "            fc2_units (int): Number of nodes in the second hidden layer\n",
    "        \"\"\"\n",
    "        super(Critic, self).__init__()\n",
    "        self.fcs1 = nn.Linear(full_state_size, fcs1_units)\n",
    "        self.fc2 = nn.Linear(fcs1_units+full_action_size, fc2_units)\n",
    "        self.fc3 = nn.Linear(fc2_units, 1)\n",
    "        self.reset_parameters()\n",
    "\n",
    "    def reset_parameters(self):\n",
    "        self.fcs1.weight.data.uniform_(*hidden_init(self.fcs1))\n",
    "        self.fc2.weight.data.uniform_(*hidden_init(self.fc2))\n",
    "        self.fc3.weight.data.uniform_(-3e-3, 3e-3)\n",
    "\n",
    "    def forward(self, full_state, full_action):\n",
    "        \"\"\"Build a critic (value) network that maps (state, action) pairs -> Q-values.\"\"\"\n",
    "        xs = F.elu(self.fcs1(full_state))\n",
    "        x = torch.cat((xs, full_action), dim=1)\n",
    "        x = F.elu(self.fc2(x))\n",
    "        return self.fc3(x)\n",
    "\n",
    "\n",
    "class MADDPG(object):\n",
    "    '''The main class that defines and trains all the agents'''\n",
    "    def __init__(self, state_size, action_size, num_agents):\n",
    "        self.state_size = state_size\n",
    "        self.action_size = action_size\n",
    "        self.num_agents = num_agents\n",
    "        self.whole_action_dim = self.action_size*self.num_agents\n",
    "        self.memory = ReplayBuffer(BUFFER_SIZE, BATCH_SIZE) # Replay memory\n",
    "        self.maddpg_agents = [DDPG(state_size, action_size, num_agents), DDPG(state_size, action_size, num_agents)] #create agents\n",
    "        self.episodes_before_training = EPISODES_BEFORE_TRAINING\n",
    "        \n",
    "    def reset(self):\n",
    "        for agent in self.maddpg_agents:\n",
    "            agent.reset()\n",
    "\n",
    "    def step(self, i_episode, states, actions, rewards, next_states, dones):\n",
    "        #for stepping maddpg\n",
    "        \"\"\"Save experience in replay memory, and use random sample from buffer to learn.\"\"\"\n",
    "        # index 0 is for agent 0 and index 1 is for agent 1\n",
    "        full_states = np.reshape(states, newshape=(-1))\n",
    "        full_next_states = np.reshape(next_states, newshape=(-1))\n",
    "        \n",
    "        # Save experience / reward\n",
    "        self.memory.add(full_states, states, actions, rewards, full_next_states, next_states, dones)\n",
    "        \n",
    "        # Learn, if enough samples are available in memory\n",
    "        if len(self.memory) > BATCH_SIZE and i_episode > self.episodes_before_training:\n",
    "            for _ in range(NUM_LEARN_STEPS_PER_ENV_STEP): #learn multiple times at every step\n",
    "                for agent_no in range(self.num_agents):\n",
    "                    samples = self.memory.sample()\n",
    "                    self.learn(samples, agent_no, GAMMA)\n",
    "                self.soft_update_all()\n",
    "\n",
    "    def soft_update_all(self):\n",
    "        #soft update all the agents            \n",
    "        for agent in self.maddpg_agents:\n",
    "            agent.soft_update_all()\n",
    "    \n",
    "    def learn(self, samples, agent_no, gamma):\n",
    "        #for learning MADDPG\n",
    "        full_states, states, actions, rewards, full_next_states, next_states, dones = samples\n",
    "        \n",
    "        critic_full_next_actions = torch.zeros(states.shape[:2] + (self.action_size,), dtype=torch.float, device=DEVICE)\n",
    "        for agent_id, agent in enumerate(self.maddpg_agents):\n",
    "            agent_next_state = next_states[:,agent_id,:]\n",
    "            critic_full_next_actions[:,agent_id,:] = agent.actor_target.forward(agent_next_state)\n",
    "        critic_full_next_actions = critic_full_next_actions.view(-1, self.whole_action_dim)\n",
    "        \n",
    "        agent = self.maddpg_agents[agent_no]\n",
    "        agent_state = states[:,agent_no,:]\n",
    "        actor_full_actions = actions.clone() #create a deep copy\n",
    "        actor_full_actions[:,agent_no,:] = agent.actor_local.forward(agent_state)\n",
    "        actor_full_actions = actor_full_actions.view(-1, self.whole_action_dim)\n",
    "                \n",
    "        full_actions = actions.view(-1,self.whole_action_dim)\n",
    "        \n",
    "        agent_rewards = rewards[:,agent_no].view(-1,1) #gives wrong result without doing this\n",
    "        agent_dones = dones[:,agent_no].view(-1,1) #gives wrong result without doing this\n",
    "        experiences = (full_states, actor_full_actions, full_actions, agent_rewards, \\\n",
    "                       agent_dones, full_next_states, critic_full_next_actions)\n",
    "        agent.learn(experiences, gamma)\n",
    "\n",
    "        #params_delete = list(agent.actor_local.parameters())\n",
    "        #print(len(params_delete))\n",
    "        #print(params_delete[2].size())\n",
    "        #print(params_delete[0][0:3,0:3])\n",
    "        \n",
    "\n",
    "    def learn_double_ddpg(self, samples, agent_no, gamma):\n",
    "        #for learning double ddpg (for debug only, do not use this)\n",
    "        full_states, states, actions, rewards, full_next_states, next_states, dones = samples\n",
    "        \n",
    "        temp_agent_no = np.random.randint(0,2) #to decide which agent's experience to use\n",
    "        strt = temp_agent_no * self.state_size\n",
    "        stp = strt + self.state_size\n",
    "        full_states = full_states[:, strt:stp]\n",
    "        states = states[:,temp_agent_no]\n",
    "        actions = actions[:,temp_agent_no]\n",
    "        rewards = rewards[:,temp_agent_no]\n",
    "        full_next_states = full_next_states[:,strt:stp]\n",
    "        next_states = next_states[:,temp_agent_no]\n",
    "        dones = dones[:,temp_agent_no]\n",
    "        \n",
    "        \n",
    "        agent = self.maddpg_agents[agent_no]\n",
    "        critic_full_next_actions = agent.actor_target.forward(next_states)\n",
    "        actor_full_actions = agent.actor_local.forward(states)\n",
    "        full_actions = actions\n",
    "        \n",
    "        experiences = (full_states, actor_full_actions, full_actions, rewards, \\\n",
    "                        dones, full_next_states, critic_full_next_actions)\n",
    "        \n",
    "        agent.learn(experiences, gamma)\n",
    "\n",
    "            \n",
    "    def act(self, full_states, i_episode, add_noise=True):\n",
    "        # all actions between -1 and 1\n",
    "        actions = []\n",
    "        for agent_id, agent in enumerate(self.maddpg_agents):\n",
    "            action = agent.act(np.reshape(full_states[agent_id,:], newshape=(1,-1)), i_episode, add_noise)\n",
    "            action = np.reshape(action, newshape=(1,-1))            \n",
    "            actions.append(action)\n",
    "        actions = np.concatenate(actions, axis=0)\n",
    "        return actions\n",
    "\n",
    "    def save_maddpg(self):\n",
    "        for agent_id, agent in enumerate(self.maddpg_agents):\n",
    "            torch.save(agent.actor_local.state_dict(), 'checkpoint_actor_local_' + str(agent_id) + '.pth')\n",
    "            torch.save(agent.critic_local.state_dict(), 'checkpoint_critic_local_' + str(agent_id) + '.pth')\n",
    "\n",
    "    def load_maddpg(self):\n",
    "        for agent_id, agent in enumerate(self.maddpg_agents):\n",
    "            #Since the model is trained on gpu, need to load all gpu tensors to cpu:\n",
    "            agent.actor_local.load_state_dict(torch.load('checkpoint_actor_local_' + str(agent_id) + '.pth', map_location=lambda storage, loc: storage))\n",
    "            agent.critic_local.load_state_dict(torch.load('checkpoint_critic_local_' + str(agent_id) + '.pth', map_location=lambda storage, loc: storage))\n",
    "\n",
    "            agent.noise_scale = NOISE_END #initialize to the final epsilon value upon training\n",
    "\n",
    "\n",
    "class DDPG(object):\n",
    "    \"\"\"Interacts with and learns from the environment.\n",
    "    There are two agents and the observations of each agent has 24 dimensions. Each agent's action has 2 dimensions.\n",
    "    Will use two separate actor networks (one for each agent using each agent's observations only and output that agent's action).\n",
    "    The critic for each agents gets to see the actions and observations of all agents. \"\"\"\n",
    "    \n",
    "    def __init__(self, state_size, action_size, num_agents):\n",
    "        \"\"\"Initialize an Agent object.\n",
    "        \n",
    "        Params\n",
    "        ======\n",
    "            state_size (int): dimension of each state for each agent\n",
    "            action_size (int): dimension of each action for each agent\n",
    "        \"\"\"\n",
    "        self.state_size = state_size\n",
    "        self.action_size = action_size        \n",
    "\n",
    "        # Actor Network (w/ Target Network)\n",
    "        self.actor_local = Actor(state_size, action_size).to(DEVICE)\n",
    "        self.actor_target = Actor(state_size, action_size).to(DEVICE)\n",
    "        self.actor_optimizer = optim.Adam(self.actor_local.parameters(), lr=LR_ACTOR, weight_decay=WEIGHT_DECAY_actor)\n",
    "\n",
    "        # Critic Network (w/ Target Network)\n",
    "        self.critic_local = Critic(num_agents*state_size, num_agents*action_size).to(DEVICE)\n",
    "        self.critic_target = Critic(num_agents*state_size, num_agents*action_size).to(DEVICE)\n",
    "        self.critic_optimizer = optim.Adam(self.critic_local.parameters(), lr=LR_CRITIC, weight_decay=WEIGHT_DECAY_critic)\n",
    "\n",
    "        # Noise process\n",
    "        self.noise = OUNoise(action_size) #single agent only\n",
    "        self.noise_scale = NOISE_START\n",
    "    \n",
    "        # Make sure target is initialized with the same weight as the source (makes a big difference)\n",
    "        self.hard_update(self.actor_target, self.actor_local)\n",
    "        self.hard_update(self.critic_target, self.critic_local)\n",
    "\n",
    "\n",
    "    def act(self, states, i_episode, add_noise=True):\n",
    "        \"\"\"Returns actions for given state as per current policy.\"\"\"\n",
    "        \n",
    "        if i_episode > EPISODES_BEFORE_TRAINING and self.noise_scale > NOISE_END:\n",
    "            #self.noise_scale *= NOISE_REDUCTION\n",
    "            self.noise_scale = NOISE_REDUCTION**(i_episode-EPISODES_BEFORE_TRAINING)\n",
    "        #else keep the previous value\n",
    "        \n",
    "        if not add_noise:\n",
    "            self.noise_scale = 0.0\n",
    "                                    \n",
    "        states = torch.from_numpy(states).float().to(DEVICE)\n",
    "        self.actor_local.eval()\n",
    "        with torch.no_grad():\n",
    "            actions = self.actor_local(states).cpu().data.numpy()\n",
    "        self.actor_local.train()\n",
    "        \n",
    "        #add noise\n",
    "        actions += self.noise_scale*self.add_noise2() #works much better than OU Noise process\n",
    "        #actions += self.noise_scale*self.noise.sample()\n",
    "        \n",
    "        return np.clip(actions, -1, 1)\n",
    "\n",
    "    def add_noise2(self):\n",
    "        noise = 0.5*np.random.randn(1,self.action_size) #sigma of 0.5 as sigma of 1 will have alot of actions just clipped\n",
    "        return noise\n",
    "        \n",
    "        \n",
    "    def reset(self):\n",
    "        self.noise.reset()\n",
    "\n",
    "    def learn(self, experiences, gamma):\n",
    "        #for MADDPG\n",
    "        \"\"\"Update policy and value parameters using given batch of experience tuples.\n",
    "        Q_targets = r + γ * critic_target(next_state, actor_target(next_state))\n",
    "        where:\n",
    "            actor_target(state) -> action\n",
    "            critic_target(state, action) -> Q-value\n",
    "\n",
    "        Params\n",
    "        ======\n",
    "            experiences (Tuple[torch.Tensor]): tuple of (s, a, r, s', done) tuples \n",
    "            gamma (float): discount factor\n",
    "        \"\"\"\n",
    "        full_states, actor_full_actions, full_actions, agent_rewards, agent_dones, full_next_states, critic_full_next_actions = experiences\n",
    "        \n",
    "        # ---------------------------- update critic ---------------------------- #\n",
    "        # Get Q values from target models\n",
    "        Q_target_next = self.critic_target(full_next_states, critic_full_next_actions)\n",
    "        # Compute Q targets for current states (y_i)\n",
    "        Q_target = agent_rewards + gamma * Q_target_next * (1 - agent_dones)\n",
    "        # Compute critic loss\n",
    "        Q_expected = self.critic_local(full_states, full_actions)\n",
    "        critic_loss = F.mse_loss(input=Q_expected, target=Q_target) #target=Q_targets.detach() #not necessary to detach\n",
    "        # Minimize the loss\n",
    "        self.critic_optimizer.zero_grad()\n",
    "        critic_loss.backward()\n",
    "        #torch.nn.utils.clip_grad_norm(self.critic_local.parameters(), 1.0) #clip the gradient for the critic network (Udacity hint)\n",
    "        self.critic_optimizer.step()\n",
    "        \n",
    "        # ---------------------------- update actor ---------------------------- #\n",
    "        # Compute actor loss\n",
    "        actor_loss = -self.critic_local.forward(full_states, actor_full_actions).mean() #-ve b'cse we want to do gradient ascent\n",
    "        # Minimize the loss\n",
    "        self.actor_optimizer.zero_grad()\n",
    "        actor_loss.backward()\n",
    "        self.actor_optimizer.step()                  \n",
    "\n",
    "        \n",
    "        \n",
    "    def soft_update_all(self):\n",
    "        # ----------------------- update target networks ----------------------- #\n",
    "        self.soft_update(self.critic_local, self.critic_target, TAU)\n",
    "        self.soft_update(self.actor_local, self.actor_target, TAU)\n",
    "\n",
    "\n",
    "   \n",
    "    def soft_update(self, local_model, target_model, tau):\n",
    "        \"\"\"Soft update model parameters.\n",
    "        θ_target = τ*θ_local + (1 - τ)*θ_target\n",
    "\n",
    "        Params\n",
    "        ======\n",
    "            local_model: PyTorch model (weights will be copied from)\n",
    "            target_model: PyTorch model (weights will be copied to)\n",
    "            tau (float): interpolation parameter \n",
    "        \"\"\"\n",
    "        for target_param, local_param in zip(target_model.parameters(), local_model.parameters()):\n",
    "            target_param.data.copy_(tau*local_param.data + (1.0-tau)*target_param.data)\n",
    "\n",
    "    def hard_update(self, target, source):\n",
    "        for target_param, source_param in zip(target.parameters(), source.parameters()):\n",
    "            target_param.data.copy_(source_param.data)\n",
    "\n",
    "\n",
    "class OUNoise(object):\n",
    "    \"\"\"Ornstein-Uhlenbeck process.\"\"\"\n",
    "\n",
    "    def __init__(self, size, mu=0., theta=0.15, sigma=0.2):\n",
    "        \"\"\"Initialize parameters and noise process.\"\"\"\n",
    "        self.mu = mu * np.ones(size)\n",
    "        self.theta = theta\n",
    "        self.sigma = sigma\n",
    "        self.size = size\n",
    "        self.reset()\n",
    "\n",
    "    def reset(self):\n",
    "        \"\"\"Reset the internal state (= noise) to mean (mu).\"\"\"\n",
    "        self.state = copy.copy(self.mu)\n",
    "\n",
    "    def sample(self):\n",
    "        \"\"\"Update internal state and return it as a noise sample.\"\"\"\n",
    "        x = self.state\n",
    "        #dx = self.theta * (self.mu - x) + self.sigma * np.array([random.random() for i in range(len(x))])\n",
    "        dx = self.theta * (self.mu - x) + self.sigma * np.random.standard_normal(self.size)\n",
    "        self.state = x + dx\n",
    "        return self.state\n",
    "\n",
    "class ReplayBuffer(object):\n",
    "    \"\"\"Fixed-size buffer to store experience tuples.\"\"\"\n",
    "#actions += self.noise.sample()\n",
    "    def __init__(self, buffer_size, batch_size):\n",
    "        \"\"\"Initialize a ReplayBuffer object.\n",
    "        Params\n",
    "        ======\n",
    "            buffer_size (int): maximum size of buffer\n",
    "            batch_size (int): size of each training batch\n",
    "        \"\"\"\n",
    "        self.memory = deque(maxlen=buffer_size)  # internal memory (deque)\n",
    "        self.batch_size = batch_size\n",
    "        self.experience = namedtuple(\"Experience\", field_names=[\"full_state\", \"state\", \"action\", \"reward\", \\\n",
    "                                                                \"full_next_state\", \"next_state\", \"done\"])\n",
    "\n",
    "    def add(self, full_state, state, action, reward, full_next_state, next_state, done):\n",
    "        \"\"\"Add a new experience to memory.\"\"\"\n",
    "        e = self.experience(full_state, state, action, reward, full_next_state, next_state, done)\n",
    "        self.memory.append(e)\n",
    "    \n",
    "    def sample(self):\n",
    "        \"\"\"Randomly sample a batch of experiences from memory.\"\"\"\n",
    "        experiences = random.sample(self.memory, k=self.batch_size)\n",
    "\n",
    "        full_states = torch.from_numpy(np.array([e.full_state for e in experiences if e is not None])).float().to(DEVICE)\n",
    "        states = torch.from_numpy(np.array([e.state for e in experiences if e is not None])).float().to(DEVICE)\n",
    "        actions = torch.from_numpy(np.array([e.action for e in experiences if e is not None])).float().to(DEVICE)\n",
    "        rewards = torch.from_numpy(np.array([e.reward for e in experiences if e is not None])).float().to(DEVICE)\n",
    "        full_next_states = torch.from_numpy(np.array([e.full_next_state for e in experiences if e is not None])).float().to(DEVICE)\n",
    "        next_states = torch.from_numpy(np.array([e.next_state for e in experiences if e is not None])).float().to(DEVICE)\n",
    "        dones = torch.from_numpy(np.array([e.done for e in experiences if e is not None]).astype(np.uint8)).float().to(DEVICE)\n",
    "\n",
    "        return (full_states, states, actions, rewards, full_next_states, next_states, dones)\n",
    "\n",
    "    def __len__(self):\n",
    "        \"\"\"Return the current size of internal memory.\"\"\"\n",
    "        return len(self.memory)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Constants\n",
    "BUFFER_SIZE = int(1e5)  # replay buffer size\n",
    "BATCH_SIZE = 256        # minibatch size\n",
    "GAMMA = 0.99            # discount factor\n",
    "TAU = 1e-3              # for soft update of target parameters\n",
    "LR_ACTOR = 1e-4 #3e-5 #1e-4         # learning rate of the actor \n",
    "LR_CRITIC = 3e-4 #3e-5 #1e-4        # learning rate of the critic\n",
    "WEIGHT_DECAY_actor = 0.0 #3e-4 #0        # L2 weight decay\n",
    "WEIGHT_DECAY_critic = 0.0 #1e-6 #0        # L2 weight decay\n",
    "#to decay exploration as it learns\n",
    "NOISE_START=1.0\n",
    "NOISE_END=0.1\n",
    "NOISE_REDUCTION=0.999\n",
    "EPISODES_BEFORE_TRAINING = 300\n",
    "NUM_LEARN_STEPS_PER_ENV_STEP = 3\n",
    "DEVICE = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
    "\n",
    "seeding(seed=2)\n",
    "state_size=env_info.vector_observations.shape[1]\n",
    "action_size=brain.vector_action_space_size\n",
    "num_agents=env_info.vector_observations.shape[0]\n",
    "MADDPG_obj = MADDPG(state_size=state_size, action_size=action_size, num_agents=num_agents)\n",
    "###MADDPG_obj.load_maddpg() #load the local network weights\n",
    "\n",
    "\n",
    "#Training\n",
    "def MADDPG_Training(n_episodes=2500, t_max=1000):\n",
    "    scores_deque = deque(maxlen=100)\n",
    "    scores_list = []\n",
    "    scores_list_100_avg = []\n",
    "    for i_episode in range(1, n_episodes+1):\n",
    "        env_info = env.reset(train_mode=True)[brain_name]     # reset the environment\n",
    "        states = env_info.vector_observations                   # get the current states (for all agents)\n",
    "        MADDPG_obj.reset() #reset the MADDPG_obj OU Noise\n",
    "        scores = np.zeros(num_agents)                          # initialize the score (for each agent in MADDPG)\n",
    "        num_steps = 0\n",
    "        for _ in range(t_max):\n",
    "            actions = MADDPG_obj.act(states, i_episode)\n",
    "            env_info = env.step(actions)[brain_name]           # send all actions to the environment\n",
    "            next_states = env_info.vector_observations         # get next state (for each agent in MADDPG)\n",
    "            rewards = env_info.rewards                         # get rewards (for each agent in MADDPG)\n",
    "            dones = env_info.local_done                        # see if episode finished\n",
    "            scores += rewards                                  # update the score (for each agent in MADDPG)\n",
    "            MADDPG_obj.step(i_episode, states, actions, rewards, next_states, dones) #train the MADDPG_obj           \n",
    "            states = next_states                               # roll over states to next time step\n",
    "            num_steps += 1\n",
    "            if np.any(dones):                                  # exit loop if episode finished\n",
    "                break\n",
    "            #print('Total score (averaged over agents) this episode: {}'.format(np.mean(score)))\n",
    "        \n",
    "        scores_deque.append(np.max(scores))\n",
    "        scores_list.append(np.max(scores))\n",
    "        scores_list_100_avg.append(np.mean(scores_deque))\n",
    "        \n",
    "        #print('\\rEpisode {}\\tAverage Score: {:.2f}\\tScore: {}'.format(i_episode, np.mean(scores_deque), score), end=\"\")\n",
    "        print('Episode {}\\tAverage Score: {:.2f}\\tCurrent Score: {}'.format(i_episode, np.mean(scores_deque), np.max(scores)))\n",
    "        print('Noise Scaling: {}, Memory size: {} and Num Steps: {}'.format(MADDPG_obj.maddpg_agents[0].noise_scale, len(MADDPG_obj.memory), num_steps))\n",
    "        \n",
    "        if i_episode % 500 == 0:\n",
    "            MADDPG_obj.save_maddpg()\n",
    "            print('Saved Model: Episode {}\\tAverage Score: {:.2f}'.format(i_episode, np.mean(scores_deque)))\n",
    "            \n",
    "        if np.mean(scores_deque) > 1.0 and len(scores_deque) >= 100:\n",
    "            MADDPG_obj.save_maddpg()\n",
    "            print('Saved Model: Episode {}\\tAverage Score: {:.2f}'.format(i_episode, np.mean(scores_deque)))\n",
    "            break\n",
    "            \n",
    "    return scores_list, scores_list_100_avg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 15 and Num Steps: 15\n",
      "Episode 2\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 29 and Num Steps: 14\n",
      "Episode 3\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 43 and Num Steps: 14\n",
      "Episode 4\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 57 and Num Steps: 14\n",
      "Episode 5\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 71 and Num Steps: 14\n",
      "Episode 6\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 86 and Num Steps: 15\n",
      "Episode 7\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 100 and Num Steps: 14\n",
      "Episode 8\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 116 and Num Steps: 16\n",
      "Episode 9\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 130 and Num Steps: 14\n",
      "Episode 10\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 144 and Num Steps: 14\n",
      "Episode 11\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 158 and Num Steps: 14\n",
      "Episode 12\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 172 and Num Steps: 14\n",
      "Episode 13\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 187 and Num Steps: 15\n",
      "Episode 14\tAverage Score: 0.01\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 1.0, Memory size: 217 and Num Steps: 30\n",
      "Episode 15\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 232 and Num Steps: 15\n",
      "Episode 16\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 246 and Num Steps: 14\n",
      "Episode 17\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 267 and Num Steps: 21\n",
      "Episode 18\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 282 and Num Steps: 15\n",
      "Episode 19\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 296 and Num Steps: 14\n",
      "Episode 20\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 310 and Num Steps: 14\n",
      "Episode 21\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 324 and Num Steps: 14\n",
      "Episode 22\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 347 and Num Steps: 23\n",
      "Episode 23\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 361 and Num Steps: 14\n",
      "Episode 24\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 375 and Num Steps: 14\n",
      "Episode 25\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 389 and Num Steps: 14\n",
      "Episode 26\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 404 and Num Steps: 15\n",
      "Episode 27\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 418 and Num Steps: 14\n",
      "Episode 28\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 434 and Num Steps: 16\n",
      "Episode 29\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 448 and Num Steps: 14\n",
      "Episode 30\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 462 and Num Steps: 14\n",
      "Episode 31\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 476 and Num Steps: 14\n",
      "Episode 32\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 490 and Num Steps: 14\n",
      "Episode 33\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 505 and Num Steps: 15\n",
      "Episode 34\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 519 and Num Steps: 14\n",
      "Episode 35\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 534 and Num Steps: 15\n",
      "Episode 36\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 548 and Num Steps: 14\n",
      "Episode 37\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 562 and Num Steps: 14\n",
      "Episode 38\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 576 and Num Steps: 14\n",
      "Episode 39\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 590 and Num Steps: 14\n",
      "Episode 40\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 605 and Num Steps: 15\n",
      "Episode 41\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 644 and Num Steps: 39\n",
      "Episode 42\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 659 and Num Steps: 15\n",
      "Episode 43\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 674 and Num Steps: 15\n",
      "Episode 44\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 688 and Num Steps: 14\n",
      "Episode 45\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 704 and Num Steps: 16\n",
      "Episode 46\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 718 and Num Steps: 14\n",
      "Episode 47\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 734 and Num Steps: 16\n",
      "Episode 48\tAverage Score: 0.01\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 1.0, Memory size: 764 and Num Steps: 30\n",
      "Episode 49\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 778 and Num Steps: 14\n",
      "Episode 50\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 792 and Num Steps: 14\n",
      "Episode 51\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 820 and Num Steps: 28\n",
      "Episode 52\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 834 and Num Steps: 14\n",
      "Episode 53\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 848 and Num Steps: 14\n",
      "Episode 54\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 863 and Num Steps: 15\n",
      "Episode 55\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 877 and Num Steps: 14\n",
      "Episode 56\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 891 and Num Steps: 14\n",
      "Episode 57\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 905 and Num Steps: 14\n",
      "Episode 58\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 920 and Num Steps: 15\n",
      "Episode 59\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 934 and Num Steps: 14\n",
      "Episode 60\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 948 and Num Steps: 14\n",
      "Episode 61\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 963 and Num Steps: 15\n",
      "Episode 62\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 977 and Num Steps: 14\n",
      "Episode 63\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 991 and Num Steps: 14\n",
      "Episode 64\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1005 and Num Steps: 14\n",
      "Episode 65\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1019 and Num Steps: 14\n",
      "Episode 66\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1034 and Num Steps: 15\n",
      "Episode 67\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1048 and Num Steps: 14\n",
      "Episode 68\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1063 and Num Steps: 15\n",
      "Episode 69\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1077 and Num Steps: 14\n",
      "Episode 70\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1091 and Num Steps: 14\n",
      "Episode 71\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1105 and Num Steps: 14\n",
      "Episode 72\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1120 and Num Steps: 15\n",
      "Episode 73\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1134 and Num Steps: 14\n",
      "Episode 74\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 1163 and Num Steps: 29\n",
      "Episode 75\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1177 and Num Steps: 14\n",
      "Episode 76\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1191 and Num Steps: 14\n",
      "Episode 77\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1205 and Num Steps: 14\n",
      "Episode 78\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1219 and Num Steps: 14\n",
      "Episode 79\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 1252 and Num Steps: 33\n",
      "Episode 80\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1266 and Num Steps: 14\n",
      "Episode 81\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1280 and Num Steps: 14\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 82\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1295 and Num Steps: 15\n",
      "Episode 83\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1309 and Num Steps: 14\n",
      "Episode 84\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1323 and Num Steps: 14\n",
      "Episode 85\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1337 and Num Steps: 14\n",
      "Episode 86\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1351 and Num Steps: 14\n",
      "Episode 87\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1372 and Num Steps: 21\n",
      "Episode 88\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1386 and Num Steps: 14\n",
      "Episode 89\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1400 and Num Steps: 14\n",
      "Episode 90\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1414 and Num Steps: 14\n",
      "Episode 91\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1428 and Num Steps: 14\n",
      "Episode 92\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1443 and Num Steps: 15\n",
      "Episode 93\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1457 and Num Steps: 14\n",
      "Episode 94\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1471 and Num Steps: 14\n",
      "Episode 95\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1485 and Num Steps: 14\n",
      "Episode 96\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1499 and Num Steps: 14\n",
      "Episode 97\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1513 and Num Steps: 14\n",
      "Episode 98\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1527 and Num Steps: 14\n",
      "Episode 99\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1542 and Num Steps: 15\n",
      "Episode 100\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1556 and Num Steps: 14\n",
      "Episode 101\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1570 and Num Steps: 14\n",
      "Episode 102\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1584 and Num Steps: 14\n",
      "Episode 103\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1598 and Num Steps: 14\n",
      "Episode 104\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1612 and Num Steps: 14\n",
      "Episode 105\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1626 and Num Steps: 14\n",
      "Episode 106\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1640 and Num Steps: 14\n",
      "Episode 107\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1654 and Num Steps: 14\n",
      "Episode 108\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1669 and Num Steps: 15\n",
      "Episode 109\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1683 and Num Steps: 14\n",
      "Episode 110\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1697 and Num Steps: 14\n",
      "Episode 111\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 1722 and Num Steps: 25\n",
      "Episode 112\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1740 and Num Steps: 18\n",
      "Episode 113\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1754 and Num Steps: 14\n",
      "Episode 114\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1769 and Num Steps: 15\n",
      "Episode 115\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1783 and Num Steps: 14\n",
      "Episode 116\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1797 and Num Steps: 14\n",
      "Episode 117\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1815 and Num Steps: 18\n",
      "Episode 118\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1829 and Num Steps: 14\n",
      "Episode 119\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1844 and Num Steps: 15\n",
      "Episode 120\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1858 and Num Steps: 14\n",
      "Episode 121\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1872 and Num Steps: 14\n",
      "Episode 122\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1886 and Num Steps: 14\n",
      "Episode 123\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1901 and Num Steps: 15\n",
      "Episode 124\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1917 and Num Steps: 16\n",
      "Episode 125\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1931 and Num Steps: 14\n",
      "Episode 126\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 1945 and Num Steps: 14\n",
      "Episode 127\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 1997 and Num Steps: 52\n",
      "Episode 128\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2011 and Num Steps: 14\n",
      "Episode 129\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2027 and Num Steps: 16\n",
      "Episode 130\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 2052 and Num Steps: 25\n",
      "Episode 131\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2072 and Num Steps: 20\n",
      "Episode 132\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2086 and Num Steps: 14\n",
      "Episode 133\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2100 and Num Steps: 14\n",
      "Episode 134\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2114 and Num Steps: 14\n",
      "Episode 135\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 2165 and Num Steps: 51\n",
      "Episode 136\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 2198 and Num Steps: 33\n",
      "Episode 137\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2212 and Num Steps: 14\n",
      "Episode 138\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 2263 and Num Steps: 51\n",
      "Episode 139\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2277 and Num Steps: 14\n",
      "Episode 140\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2296 and Num Steps: 19\n",
      "Episode 141\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2310 and Num Steps: 14\n",
      "Episode 142\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2324 and Num Steps: 14\n",
      "Episode 143\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2338 and Num Steps: 14\n",
      "Episode 144\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2353 and Num Steps: 15\n",
      "Episode 145\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2367 and Num Steps: 14\n",
      "Episode 146\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2381 and Num Steps: 14\n",
      "Episode 147\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2395 and Num Steps: 14\n",
      "Episode 148\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2409 and Num Steps: 14\n",
      "Episode 149\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2424 and Num Steps: 15\n",
      "Episode 150\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2449 and Num Steps: 25\n",
      "Episode 151\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2464 and Num Steps: 15\n",
      "Episode 152\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2479 and Num Steps: 15\n",
      "Episode 153\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2493 and Num Steps: 14\n",
      "Episode 154\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2507 and Num Steps: 14\n",
      "Episode 155\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2521 and Num Steps: 14\n",
      "Episode 156\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2535 and Num Steps: 14\n",
      "Episode 157\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2549 and Num Steps: 14\n",
      "Episode 158\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2563 and Num Steps: 14\n",
      "Episode 159\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2577 and Num Steps: 14\n",
      "Episode 160\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2591 and Num Steps: 14\n",
      "Episode 161\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2606 and Num Steps: 15\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 162\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2621 and Num Steps: 15\n",
      "Episode 163\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2635 and Num Steps: 14\n",
      "Episode 164\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2652 and Num Steps: 17\n",
      "Episode 165\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2666 and Num Steps: 14\n",
      "Episode 166\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2680 and Num Steps: 14\n",
      "Episode 167\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2694 and Num Steps: 14\n",
      "Episode 168\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2708 and Num Steps: 14\n",
      "Episode 169\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 2761 and Num Steps: 53\n",
      "Episode 170\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2775 and Num Steps: 14\n",
      "Episode 171\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2789 and Num Steps: 14\n",
      "Episode 172\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2804 and Num Steps: 15\n",
      "Episode 173\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 2849 and Num Steps: 45\n",
      "Episode 174\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2863 and Num Steps: 14\n",
      "Episode 175\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2877 and Num Steps: 14\n",
      "Episode 176\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2891 and Num Steps: 14\n",
      "Episode 177\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2906 and Num Steps: 15\n",
      "Episode 178\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2920 and Num Steps: 14\n",
      "Episode 179\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2934 and Num Steps: 14\n",
      "Episode 180\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2948 and Num Steps: 14\n",
      "Episode 181\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 2962 and Num Steps: 14\n",
      "Episode 182\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 3013 and Num Steps: 51\n",
      "Episode 183\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3029 and Num Steps: 16\n",
      "Episode 184\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3044 and Num Steps: 15\n",
      "Episode 185\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3058 and Num Steps: 14\n",
      "Episode 186\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3072 and Num Steps: 14\n",
      "Episode 187\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3086 and Num Steps: 14\n",
      "Episode 188\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3103 and Num Steps: 17\n",
      "Episode 189\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3119 and Num Steps: 16\n",
      "Episode 190\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3133 and Num Steps: 14\n",
      "Episode 191\tAverage Score: 0.01\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 1.0, Memory size: 3163 and Num Steps: 30\n",
      "Episode 192\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3177 and Num Steps: 14\n",
      "Episode 193\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3193 and Num Steps: 16\n",
      "Episode 194\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 3226 and Num Steps: 33\n",
      "Episode 195\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3240 and Num Steps: 14\n",
      "Episode 196\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3255 and Num Steps: 15\n",
      "Episode 197\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3269 and Num Steps: 14\n",
      "Episode 198\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3285 and Num Steps: 16\n",
      "Episode 199\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 3318 and Num Steps: 33\n",
      "Episode 200\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3333 and Num Steps: 15\n",
      "Episode 201\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3347 and Num Steps: 14\n",
      "Episode 202\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 3369 and Num Steps: 22\n",
      "Episode 203\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3383 and Num Steps: 14\n",
      "Episode 204\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3397 and Num Steps: 14\n",
      "Episode 205\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3411 and Num Steps: 14\n",
      "Episode 206\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3426 and Num Steps: 15\n",
      "Episode 207\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3440 and Num Steps: 14\n",
      "Episode 208\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3454 and Num Steps: 14\n",
      "Episode 209\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 3485 and Num Steps: 31\n",
      "Episode 210\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3499 and Num Steps: 14\n",
      "Episode 211\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3513 and Num Steps: 14\n",
      "Episode 212\tAverage Score: 0.01\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 1.0, Memory size: 3544 and Num Steps: 31\n",
      "Episode 213\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3558 and Num Steps: 14\n",
      "Episode 214\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 3588 and Num Steps: 30\n",
      "Episode 215\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 3639 and Num Steps: 51\n",
      "Episode 216\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3654 and Num Steps: 15\n",
      "Episode 217\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3668 and Num Steps: 14\n",
      "Episode 218\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3682 and Num Steps: 14\n",
      "Episode 219\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 3710 and Num Steps: 28\n",
      "Episode 220\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3724 and Num Steps: 14\n",
      "Episode 221\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 3754 and Num Steps: 30\n",
      "Episode 222\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 3787 and Num Steps: 33\n",
      "Episode 223\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3801 and Num Steps: 14\n",
      "Episode 224\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3816 and Num Steps: 15\n",
      "Episode 225\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3830 and Num Steps: 14\n",
      "Episode 226\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3844 and Num Steps: 14\n",
      "Episode 227\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 3891 and Num Steps: 47\n",
      "Episode 228\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3905 and Num Steps: 14\n",
      "Episode 229\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3920 and Num Steps: 15\n",
      "Episode 230\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3934 and Num Steps: 14\n",
      "Episode 231\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 3978 and Num Steps: 44\n",
      "Episode 232\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 3992 and Num Steps: 14\n",
      "Episode 233\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 4017 and Num Steps: 25\n",
      "Episode 234\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4031 and Num Steps: 14\n",
      "Episode 235\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4045 and Num Steps: 14\n",
      "Episode 236\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4065 and Num Steps: 20\n",
      "Episode 237\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4080 and Num Steps: 15\n",
      "Episode 238\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4094 and Num Steps: 14\n",
      "Episode 239\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4108 and Num Steps: 14\n",
      "Episode 240\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4122 and Num Steps: 14\n",
      "Episode 241\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4136 and Num Steps: 14\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 242\tAverage Score: 0.02\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 1.0, Memory size: 4229 and Num Steps: 93\n",
      "Episode 243\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4243 and Num Steps: 14\n",
      "Episode 244\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 4272 and Num Steps: 29\n",
      "Episode 245\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4286 and Num Steps: 14\n",
      "Episode 246\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 4336 and Num Steps: 50\n",
      "Episode 247\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4350 and Num Steps: 14\n",
      "Episode 248\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4364 and Num Steps: 14\n",
      "Episode 249\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4382 and Num Steps: 18\n",
      "Episode 250\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4396 and Num Steps: 14\n",
      "Episode 251\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4410 and Num Steps: 14\n",
      "Episode 252\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4424 and Num Steps: 14\n",
      "Episode 253\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4438 and Num Steps: 14\n",
      "Episode 254\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4452 and Num Steps: 14\n",
      "Episode 255\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 4475 and Num Steps: 23\n",
      "Episode 256\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4489 and Num Steps: 14\n",
      "Episode 257\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4503 and Num Steps: 14\n",
      "Episode 258\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4517 and Num Steps: 14\n",
      "Episode 259\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 4547 and Num Steps: 30\n",
      "Episode 260\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 4577 and Num Steps: 30\n",
      "Episode 261\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4591 and Num Steps: 14\n",
      "Episode 262\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4606 and Num Steps: 15\n",
      "Episode 263\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4620 and Num Steps: 14\n",
      "Episode 264\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4634 and Num Steps: 14\n",
      "Episode 265\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 4667 and Num Steps: 33\n",
      "Episode 266\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4681 and Num Steps: 14\n",
      "Episode 267\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4696 and Num Steps: 15\n",
      "Episode 268\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4714 and Num Steps: 18\n",
      "Episode 269\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4730 and Num Steps: 16\n",
      "Episode 270\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4745 and Num Steps: 15\n",
      "Episode 271\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4759 and Num Steps: 14\n",
      "Episode 272\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4773 and Num Steps: 14\n",
      "Episode 273\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4787 and Num Steps: 14\n",
      "Episode 274\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4801 and Num Steps: 14\n",
      "Episode 275\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4815 and Num Steps: 14\n",
      "Episode 276\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4829 and Num Steps: 14\n",
      "Episode 277\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4848 and Num Steps: 19\n",
      "Episode 278\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4863 and Num Steps: 15\n",
      "Episode 279\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4878 and Num Steps: 15\n",
      "Episode 280\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4892 and Num Steps: 14\n",
      "Episode 281\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4906 and Num Steps: 14\n",
      "Episode 282\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4920 and Num Steps: 14\n",
      "Episode 283\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4936 and Num Steps: 16\n",
      "Episode 284\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4950 and Num Steps: 14\n",
      "Episode 285\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 1.0, Memory size: 4976 and Num Steps: 26\n",
      "Episode 286\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 4990 and Num Steps: 14\n",
      "Episode 287\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 5004 and Num Steps: 14\n",
      "Episode 288\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 5019 and Num Steps: 15\n",
      "Episode 289\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 5033 and Num Steps: 14\n",
      "Episode 290\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 5047 and Num Steps: 14\n",
      "Episode 291\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 5061 and Num Steps: 14\n",
      "Episode 292\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 5075 and Num Steps: 14\n",
      "Episode 293\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 5090 and Num Steps: 15\n",
      "Episode 294\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 5104 and Num Steps: 14\n",
      "Episode 295\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 5118 and Num Steps: 14\n",
      "Episode 296\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 5133 and Num Steps: 15\n",
      "Episode 297\tAverage Score: 0.02\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 1.0, Memory size: 5163 and Num Steps: 30\n",
      "Episode 298\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 5176 and Num Steps: 13\n",
      "Episode 299\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 5191 and Num Steps: 15\n",
      "Episode 300\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 1.0, Memory size: 5205 and Num Steps: 14\n",
      "Episode 301\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.999, Memory size: 5219 and Num Steps: 14\n",
      "Episode 302\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.998001, Memory size: 5233 and Num Steps: 14\n",
      "Episode 303\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.997002999, Memory size: 5247 and Num Steps: 14\n",
      "Episode 304\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.996005996001, Memory size: 5262 and Num Steps: 15\n",
      "Episode 305\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.995009990004999, Memory size: 5276 and Num Steps: 14\n",
      "Episode 306\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.994014980014994, Memory size: 5290 and Num Steps: 14\n",
      "Episode 307\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.993020965034979, Memory size: 5304 and Num Steps: 14\n",
      "Episode 308\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.992027944069944, Memory size: 5318 and Num Steps: 14\n",
      "Episode 309\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.991035916125874, Memory size: 5333 and Num Steps: 15\n",
      "Episode 310\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9900448802097482, Memory size: 5347 and Num Steps: 14\n",
      "Episode 311\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9890548353295384, Memory size: 5361 and Num Steps: 14\n",
      "Episode 312\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9880657804942089, Memory size: 5375 and Num Steps: 14\n",
      "Episode 313\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9870777147137147, Memory size: 5389 and Num Steps: 14\n",
      "Episode 314\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.986090636999001, Memory size: 5404 and Num Steps: 15\n",
      "Episode 315\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.985104546362002, Memory size: 5418 and Num Steps: 14\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 316\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9841194418156399, Memory size: 5432 and Num Steps: 14\n",
      "Episode 317\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9831353223738244, Memory size: 5446 and Num Steps: 14\n",
      "Episode 318\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9821521870514505, Memory size: 5460 and Num Steps: 14\n",
      "Episode 319\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.981170034864399, Memory size: 5475 and Num Steps: 15\n",
      "Episode 320\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9801888648295347, Memory size: 5489 and Num Steps: 14\n",
      "Episode 321\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9792086759647052, Memory size: 5503 and Num Steps: 14\n",
      "Episode 322\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9782294672887404, Memory size: 5517 and Num Steps: 14\n",
      "Episode 323\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9772512378214517, Memory size: 5531 and Num Steps: 14\n",
      "Episode 324\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9762739865836302, Memory size: 5546 and Num Steps: 15\n",
      "Episode 325\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9752977125970466, Memory size: 5560 and Num Steps: 14\n",
      "Episode 326\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9743224148844496, Memory size: 5574 and Num Steps: 14\n",
      "Episode 327\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9733480924695651, Memory size: 5588 and Num Steps: 14\n",
      "Episode 328\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9723747443770956, Memory size: 5602 and Num Steps: 14\n",
      "Episode 329\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9714023696327184, Memory size: 5617 and Num Steps: 15\n",
      "Episode 330\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9704309672630858, Memory size: 5631 and Num Steps: 14\n",
      "Episode 331\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9694605362958226, Memory size: 5645 and Num Steps: 14\n",
      "Episode 332\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9684910757595268, Memory size: 5659 and Num Steps: 14\n",
      "Episode 333\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9675225846837673, Memory size: 5673 and Num Steps: 14\n",
      "Episode 334\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9665550620990835, Memory size: 5688 and Num Steps: 15\n",
      "Episode 335\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9655885070369844, Memory size: 5702 and Num Steps: 14\n",
      "Episode 336\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9646229185299474, Memory size: 5716 and Num Steps: 14\n",
      "Episode 337\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9636582956114175, Memory size: 5730 and Num Steps: 14\n",
      "Episode 338\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9626946373158061, Memory size: 5744 and Num Steps: 14\n",
      "Episode 339\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9617319426784903, Memory size: 5759 and Num Steps: 15\n",
      "Episode 340\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9607702107358118, Memory size: 5773 and Num Steps: 14\n",
      "Episode 341\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.959809440525076, Memory size: 5787 and Num Steps: 14\n",
      "Episode 342\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9588496310845509, Memory size: 5801 and Num Steps: 14\n",
      "Episode 343\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9578907814534663, Memory size: 5815 and Num Steps: 14\n",
      "Episode 344\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9569328906720129, Memory size: 5830 and Num Steps: 15\n",
      "Episode 345\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9559759577813408, Memory size: 5844 and Num Steps: 14\n",
      "Episode 346\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9550199818235595, Memory size: 5858 and Num Steps: 14\n",
      "Episode 347\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.954064961841736, Memory size: 5872 and Num Steps: 14\n",
      "Episode 348\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9531108968798943, Memory size: 5886 and Num Steps: 14\n",
      "Episode 349\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9521577859830144, Memory size: 5901 and Num Steps: 15\n",
      "Episode 350\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9512056281970314, Memory size: 5915 and Num Steps: 14\n",
      "Episode 351\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9502544225688343, Memory size: 5929 and Num Steps: 14\n",
      "Episode 352\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9493041681462654, Memory size: 5943 and Num Steps: 14\n",
      "Episode 353\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9483548639781192, Memory size: 5957 and Num Steps: 14\n",
      "Episode 354\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.947406509114141, Memory size: 5972 and Num Steps: 15\n",
      "Episode 355\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9464591026050269, Memory size: 5986 and Num Steps: 14\n",
      "Episode 356\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9455126435024219, Memory size: 6000 and Num Steps: 14\n",
      "Episode 357\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9445671308589194, Memory size: 6014 and Num Steps: 14\n",
      "Episode 358\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9436225637280605, Memory size: 6028 and Num Steps: 14\n",
      "Episode 359\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9426789411643325, Memory size: 6043 and Num Steps: 15\n",
      "Episode 360\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9417362622231682, Memory size: 6057 and Num Steps: 14\n",
      "Episode 361\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9407945259609449, Memory size: 6071 and Num Steps: 14\n",
      "Episode 362\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9398537314349841, Memory size: 6085 and Num Steps: 14\n",
      "Episode 363\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9389138777035491, Memory size: 6099 and Num Steps: 14\n",
      "Episode 364\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9379749638258454, Memory size: 6114 and Num Steps: 15\n",
      "Episode 365\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9370369888620197, Memory size: 6128 and Num Steps: 14\n",
      "Episode 366\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9360999518731576, Memory size: 6142 and Num Steps: 14\n",
      "Episode 367\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9351638519212845, Memory size: 6156 and Num Steps: 14\n",
      "Episode 368\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9342286880693632, Memory size: 6170 and Num Steps: 14\n",
      "Episode 369\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9332944593812939, Memory size: 6185 and Num Steps: 15\n",
      "Episode 370\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9323611649219126, Memory size: 6199 and Num Steps: 14\n",
      "Episode 371\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9314288037569907, Memory size: 6213 and Num Steps: 14\n",
      "Episode 372\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9304973749532336, Memory size: 6227 and Num Steps: 14\n",
      "Episode 373\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9295668775782804, Memory size: 6241 and Num Steps: 14\n",
      "Episode 374\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9286373107007021, Memory size: 6256 and Num Steps: 15\n",
      "Episode 375\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9277086733900014, Memory size: 6270 and Num Steps: 14\n",
      "Episode 376\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9267809647166114, Memory size: 6284 and Num Steps: 14\n",
      "Episode 377\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9258541837518948, Memory size: 6298 and Num Steps: 14\n",
      "Episode 378\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9249283295681429, Memory size: 6312 and Num Steps: 14\n",
      "Episode 379\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9240034012385747, Memory size: 6327 and Num Steps: 15\n",
      "Episode 380\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9230793978373362, Memory size: 6341 and Num Steps: 14\n",
      "Episode 381\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9221563184394989, Memory size: 6355 and Num Steps: 14\n",
      "Episode 382\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9212341621210594, Memory size: 6369 and Num Steps: 14\n",
      "Episode 383\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9203129279589383, Memory size: 6383 and Num Steps: 14\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 384\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9193926150309794, Memory size: 6398 and Num Steps: 15\n",
      "Episode 385\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9184732224159483, Memory size: 6412 and Num Steps: 14\n",
      "Episode 386\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9175547491935324, Memory size: 6426 and Num Steps: 14\n",
      "Episode 387\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9166371944443389, Memory size: 6440 and Num Steps: 14\n",
      "Episode 388\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9157205572498945, Memory size: 6454 and Num Steps: 14\n",
      "Episode 389\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9148048366926447, Memory size: 6469 and Num Steps: 15\n",
      "Episode 390\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.913890031855952, Memory size: 6483 and Num Steps: 14\n",
      "Episode 391\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.912976141824096, Memory size: 6497 and Num Steps: 14\n",
      "Episode 392\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.912063165682272, Memory size: 6511 and Num Steps: 14\n",
      "Episode 393\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9111511025165897, Memory size: 6525 and Num Steps: 14\n",
      "Episode 394\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9102399514140731, Memory size: 6540 and Num Steps: 15\n",
      "Episode 395\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9093297114626591, Memory size: 6554 and Num Steps: 14\n",
      "Episode 396\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9084203817511963, Memory size: 6568 and Num Steps: 14\n",
      "Episode 397\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9075119613694451, Memory size: 6582 and Num Steps: 14\n",
      "Episode 398\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9066044494080757, Memory size: 6596 and Num Steps: 14\n",
      "Episode 399\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9056978449586677, Memory size: 6614 and Num Steps: 18\n",
      "Episode 400\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9047921471137089, Memory size: 6628 and Num Steps: 14\n",
      "Episode 401\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9038873549665952, Memory size: 6642 and Num Steps: 14\n",
      "Episode 402\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9029834676116286, Memory size: 6656 and Num Steps: 14\n",
      "Episode 403\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.902080484144017, Memory size: 6671 and Num Steps: 15\n",
      "Episode 404\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.901178403659873, Memory size: 6685 and Num Steps: 14\n",
      "Episode 405\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.9002772252562131, Memory size: 6699 and Num Steps: 14\n",
      "Episode 406\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8993769480309569, Memory size: 6714 and Num Steps: 15\n",
      "Episode 407\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8984775710829259, Memory size: 6728 and Num Steps: 14\n",
      "Episode 408\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8975790935118431, Memory size: 6742 and Num Steps: 14\n",
      "Episode 409\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8966815144183312, Memory size: 6756 and Num Steps: 14\n",
      "Episode 410\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8957848329039129, Memory size: 6770 and Num Steps: 14\n",
      "Episode 411\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.894889048071009, Memory size: 6785 and Num Steps: 15\n",
      "Episode 412\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8939941590229379, Memory size: 6799 and Num Steps: 14\n",
      "Episode 413\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.893100164863915, Memory size: 6813 and Num Steps: 14\n",
      "Episode 414\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8922070646990511, Memory size: 6827 and Num Steps: 14\n",
      "Episode 415\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.891314857634352, Memory size: 6841 and Num Steps: 14\n",
      "Episode 416\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8904235427767176, Memory size: 6856 and Num Steps: 15\n",
      "Episode 417\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.889533119233941, Memory size: 6870 and Num Steps: 14\n",
      "Episode 418\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8886435861147071, Memory size: 6884 and Num Steps: 14\n",
      "Episode 419\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8877549425285923, Memory size: 6898 and Num Steps: 14\n",
      "Episode 420\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8868671875860638, Memory size: 6912 and Num Steps: 14\n",
      "Episode 421\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8859803203984776, Memory size: 6927 and Num Steps: 15\n",
      "Episode 422\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8850943400780792, Memory size: 6941 and Num Steps: 14\n",
      "Episode 423\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8842092457380011, Memory size: 6955 and Num Steps: 14\n",
      "Episode 424\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8833250364922631, Memory size: 6969 and Num Steps: 14\n",
      "Episode 425\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8824417114557709, Memory size: 6983 and Num Steps: 14\n",
      "Episode 426\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8815592697443151, Memory size: 6998 and Num Steps: 15\n",
      "Episode 427\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8806777104745708, Memory size: 7012 and Num Steps: 14\n",
      "Episode 428\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8797970327640962, Memory size: 7026 and Num Steps: 14\n",
      "Episode 429\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8789172357313321, Memory size: 7040 and Num Steps: 14\n",
      "Episode 430\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8780383184956008, Memory size: 7054 and Num Steps: 14\n",
      "Episode 431\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8771602801771051, Memory size: 7069 and Num Steps: 15\n",
      "Episode 432\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8762831198969281, Memory size: 7083 and Num Steps: 14\n",
      "Episode 433\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8754068367770311, Memory size: 7097 and Num Steps: 14\n",
      "Episode 434\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8745314299402541, Memory size: 7111 and Num Steps: 14\n",
      "Episode 435\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8736568985103138, Memory size: 7125 and Num Steps: 14\n",
      "Episode 436\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8727832416118035, Memory size: 7140 and Num Steps: 15\n",
      "Episode 437\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8719104583701917, Memory size: 7154 and Num Steps: 14\n",
      "Episode 438\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8710385479118216, Memory size: 7168 and Num Steps: 14\n",
      "Episode 439\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8701675093639096, Memory size: 7182 and Num Steps: 14\n",
      "Episode 440\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8692973418545458, Memory size: 7196 and Num Steps: 14\n",
      "Episode 441\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8684280445126912, Memory size: 7211 and Num Steps: 15\n",
      "Episode 442\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8675596164681786, Memory size: 7225 and Num Steps: 14\n",
      "Episode 443\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8666920568517104, Memory size: 7239 and Num Steps: 14\n",
      "Episode 444\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8658253647948586, Memory size: 7253 and Num Steps: 14\n",
      "Episode 445\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8649595394300638, Memory size: 7267 and Num Steps: 14\n",
      "Episode 446\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8640945798906338, Memory size: 7282 and Num Steps: 15\n",
      "Episode 447\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8632304853107431, Memory size: 7296 and Num Steps: 14\n",
      "Episode 448\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8623672548254323, Memory size: 7310 and Num Steps: 14\n",
      "Episode 449\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.861504887570607, Memory size: 7324 and Num Steps: 14\n",
      "Episode 450\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8606433826830363, Memory size: 7338 and Num Steps: 14\n",
      "Episode 451\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8597827393003533, Memory size: 7353 and Num Steps: 15\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 452\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8589229565610529, Memory size: 7367 and Num Steps: 14\n",
      "Episode 453\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8580640336044919, Memory size: 7381 and Num Steps: 14\n",
      "Episode 454\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8572059695708873, Memory size: 7395 and Num Steps: 14\n",
      "Episode 455\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8563487636013165, Memory size: 7409 and Num Steps: 14\n",
      "Episode 456\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8554924148377152, Memory size: 7424 and Num Steps: 15\n",
      "Episode 457\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8546369224228775, Memory size: 7439 and Num Steps: 15\n",
      "Episode 458\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8537822855004545, Memory size: 7453 and Num Steps: 14\n",
      "Episode 459\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8529285032149541, Memory size: 7467 and Num Steps: 14\n",
      "Episode 460\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8520755747117391, Memory size: 7481 and Num Steps: 14\n",
      "Episode 461\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8512234991370274, Memory size: 7495 and Num Steps: 14\n",
      "Episode 462\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8503722756378904, Memory size: 7510 and Num Steps: 15\n",
      "Episode 463\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8495219033622525, Memory size: 7524 and Num Steps: 14\n",
      "Episode 464\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8486723814588902, Memory size: 7538 and Num Steps: 14\n",
      "Episode 465\tAverage Score: 0.00\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.8478237090774313, Memory size: 7568 and Num Steps: 30\n",
      "Episode 466\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8469758853683539, Memory size: 7582 and Num Steps: 14\n",
      "Episode 467\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8461289094829856, Memory size: 7597 and Num Steps: 15\n",
      "Episode 468\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8452827805735026, Memory size: 7613 and Num Steps: 16\n",
      "Episode 469\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8444374977929291, Memory size: 7627 and Num Steps: 14\n",
      "Episode 470\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8435930602951361, Memory size: 7641 and Num Steps: 14\n",
      "Episode 471\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.842749467234841, Memory size: 7655 and Num Steps: 14\n",
      "Episode 472\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8419067177676062, Memory size: 7669 and Num Steps: 14\n",
      "Episode 473\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8410648110498385, Memory size: 7683 and Num Steps: 14\n",
      "Episode 474\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8402237462387887, Memory size: 7697 and Num Steps: 14\n",
      "Episode 475\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8393835224925499, Memory size: 7712 and Num Steps: 15\n",
      "Episode 476\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8385441389700574, Memory size: 7728 and Num Steps: 16\n",
      "Episode 477\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8377055948310873, Memory size: 7743 and Num Steps: 15\n",
      "Episode 478\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8368678892362562, Memory size: 7757 and Num Steps: 14\n",
      "Episode 479\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.83603102134702, Memory size: 7771 and Num Steps: 14\n",
      "Episode 480\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8351949903256729, Memory size: 7786 and Num Steps: 15\n",
      "Episode 481\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8343597953353473, Memory size: 7800 and Num Steps: 14\n",
      "Episode 482\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8335254355400119, Memory size: 7814 and Num Steps: 14\n",
      "Episode 483\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.832691910104472, Memory size: 7827 and Num Steps: 13\n",
      "Episode 484\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8318592181943675, Memory size: 7846 and Num Steps: 19\n",
      "Episode 485\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8310273589761731, Memory size: 7861 and Num Steps: 15\n",
      "Episode 486\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8301963316171969, Memory size: 7876 and Num Steps: 15\n",
      "Episode 487\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8293661352855797, Memory size: 7890 and Num Steps: 14\n",
      "Episode 488\tAverage Score: 0.00\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.8285367691502942, Memory size: 7923 and Num Steps: 33\n",
      "Episode 489\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8277082323811439, Memory size: 7942 and Num Steps: 19\n",
      "Episode 490\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8268805241487627, Memory size: 7956 and Num Steps: 14\n",
      "Episode 491\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.826053643624614, Memory size: 7970 and Num Steps: 14\n",
      "Episode 492\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8252275899809893, Memory size: 7984 and Num Steps: 14\n",
      "Episode 493\tAverage Score: 0.00\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.8244023623910083, Memory size: 8017 and Num Steps: 33\n",
      "Episode 494\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8235779600286173, Memory size: 8031 and Num Steps: 14\n",
      "Episode 495\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8227543820685886, Memory size: 8046 and Num Steps: 15\n",
      "Episode 496\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8219316276865201, Memory size: 8060 and Num Steps: 14\n",
      "Episode 497\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8211096960588335, Memory size: 8074 and Num Steps: 14\n",
      "Episode 498\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8202885863627748, Memory size: 8088 and Num Steps: 14\n",
      "Episode 499\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.819468297776412, Memory size: 8102 and Num Steps: 14\n",
      "Episode 500\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8186488294786356, Memory size: 8121 and Num Steps: 19\n",
      "Saved Model: Episode 500\tAverage Score: 0.00\n",
      "Episode 501\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8178301806491569, Memory size: 8135 and Num Steps: 14\n",
      "Episode 502\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8170123504685077, Memory size: 8150 and Num Steps: 15\n",
      "Episode 503\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8161953381180392, Memory size: 8164 and Num Steps: 14\n",
      "Episode 504\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8153791427799212, Memory size: 8178 and Num Steps: 14\n",
      "Episode 505\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8145637636371413, Memory size: 8192 and Num Steps: 14\n",
      "Episode 506\tAverage Score: 0.00\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.8137491998735041, Memory size: 8225 and Num Steps: 33\n",
      "Episode 507\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8129354506736306, Memory size: 8239 and Num Steps: 14\n",
      "Episode 508\tAverage Score: 0.00\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.812122515222957, Memory size: 8253 and Num Steps: 14\n",
      "Episode 509\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.8113103927077341, Memory size: 8286 and Num Steps: 33\n",
      "Episode 510\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8104990823150263, Memory size: 8302 and Num Steps: 16\n",
      "Episode 511\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8096885832327113, Memory size: 8316 and Num Steps: 14\n",
      "Episode 512\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8088788946494786, Memory size: 8330 and Num Steps: 14\n",
      "Episode 513\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8080700157548291, Memory size: 8348 and Num Steps: 18\n",
      "Episode 514\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8072619457390743, Memory size: 8363 and Num Steps: 15\n",
      "Episode 515\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.8064546837933352, Memory size: 8395 and Num Steps: 32\n",
      "Episode 516\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8056482291095418, Memory size: 8428 and Num Steps: 33\n",
      "Episode 517\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8048425808804324, Memory size: 8442 and Num Steps: 14\n",
      "Episode 518\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8040377382995519, Memory size: 8460 and Num Steps: 18\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 519\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8032337005612523, Memory size: 8474 and Num Steps: 14\n",
      "Episode 520\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8024304668606911, Memory size: 8492 and Num Steps: 18\n",
      "Episode 521\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.8016280363938304, Memory size: 8524 and Num Steps: 32\n",
      "Episode 522\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8008264083574366, Memory size: 8538 and Num Steps: 14\n",
      "Episode 523\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.8000255819490791, Memory size: 8553 and Num Steps: 15\n",
      "Episode 524\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.79922555636713, Memory size: 8567 and Num Steps: 14\n",
      "Episode 525\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7984263308107629, Memory size: 8600 and Num Steps: 33\n",
      "Episode 526\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7976279044799521, Memory size: 8615 and Num Steps: 15\n",
      "Episode 527\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7968302765754722, Memory size: 8629 and Num Steps: 14\n",
      "Episode 528\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7960334462988967, Memory size: 8661 and Num Steps: 32\n",
      "Episode 529\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7952374128525979, Memory size: 8676 and Num Steps: 15\n",
      "Episode 530\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7944421754397453, Memory size: 8690 and Num Steps: 14\n",
      "Episode 531\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7936477332643055, Memory size: 8723 and Num Steps: 33\n",
      "Episode 532\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7928540855310412, Memory size: 8737 and Num Steps: 14\n",
      "Episode 533\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7920612314455101, Memory size: 8751 and Num Steps: 14\n",
      "Episode 534\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7912691702140646, Memory size: 8765 and Num Steps: 14\n",
      "Episode 535\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7904779010438505, Memory size: 8783 and Num Steps: 18\n",
      "Episode 536\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7896874231428067, Memory size: 8799 and Num Steps: 16\n",
      "Episode 537\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7888977357196639, Memory size: 8817 and Num Steps: 18\n",
      "Episode 538\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7881088379839443, Memory size: 8831 and Num Steps: 14\n",
      "Episode 539\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7873207291459603, Memory size: 8846 and Num Steps: 15\n",
      "Episode 540\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7865334084168143, Memory size: 8860 and Num Steps: 14\n",
      "Episode 541\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7857468750083975, Memory size: 8893 and Num Steps: 33\n",
      "Episode 542\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7849611281333891, Memory size: 8907 and Num Steps: 14\n",
      "Episode 543\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7841761670052557, Memory size: 8921 and Num Steps: 14\n",
      "Episode 544\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7833919908382505, Memory size: 8935 and Num Steps: 14\n",
      "Episode 545\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7826085988474122, Memory size: 8950 and Num Steps: 15\n",
      "Episode 546\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7818259902485648, Memory size: 8964 and Num Steps: 14\n",
      "Episode 547\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7810441642583162, Memory size: 8982 and Num Steps: 18\n",
      "Episode 548\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.780263120094058, Memory size: 8996 and Num Steps: 14\n",
      "Episode 549\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7794828569739639, Memory size: 9010 and Num Steps: 14\n",
      "Episode 550\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7787033741169899, Memory size: 9025 and Num Steps: 15\n",
      "Episode 551\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7779246707428729, Memory size: 9039 and Num Steps: 14\n",
      "Episode 552\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7771467460721301, Memory size: 9053 and Num Steps: 14\n",
      "Episode 553\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7763695993260579, Memory size: 9067 and Num Steps: 14\n",
      "Episode 554\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7755932297267318, Memory size: 9094 and Num Steps: 27\n",
      "Episode 555\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7748176364970051, Memory size: 9108 and Num Steps: 14\n",
      "Episode 556\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7740428188605081, Memory size: 9122 and Num Steps: 14\n",
      "Episode 557\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7732687760416476, Memory size: 9137 and Num Steps: 15\n",
      "Episode 558\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7724955072656059, Memory size: 9151 and Num Steps: 14\n",
      "Episode 559\tAverage Score: 0.01\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.7717230117583403, Memory size: 9181 and Num Steps: 30\n",
      "Episode 560\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.770951288746582, Memory size: 9195 and Num Steps: 14\n",
      "Episode 561\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7701803374578354, Memory size: 9210 and Num Steps: 15\n",
      "Episode 562\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7694101571203776, Memory size: 9224 and Num Steps: 14\n",
      "Episode 563\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7686407469632572, Memory size: 9257 and Num Steps: 33\n",
      "Episode 564\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.767872106216294, Memory size: 9271 and Num Steps: 14\n",
      "Episode 565\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7671042341100777, Memory size: 9285 and Num Steps: 14\n",
      "Episode 566\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7663371298759676, Memory size: 9299 and Num Steps: 14\n",
      "Episode 567\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7655707927460916, Memory size: 9332 and Num Steps: 33\n",
      "Episode 568\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7648052219533455, Memory size: 9350 and Num Steps: 18\n",
      "Episode 569\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7640404167313921, Memory size: 9364 and Num Steps: 14\n",
      "Episode 570\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7632763763146607, Memory size: 9378 and Num Steps: 14\n",
      "Episode 571\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7625130999383462, Memory size: 9393 and Num Steps: 15\n",
      "Episode 572\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7617505868384078, Memory size: 9410 and Num Steps: 17\n",
      "Episode 573\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7609888362515693, Memory size: 9425 and Num Steps: 15\n",
      "Episode 574\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7602278474153178, Memory size: 9457 and Num Steps: 32\n",
      "Episode 575\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7594676195679024, Memory size: 9471 and Num Steps: 14\n",
      "Episode 576\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7587081519483345, Memory size: 9485 and Num Steps: 14\n",
      "Episode 577\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7579494437963862, Memory size: 9499 and Num Steps: 14\n",
      "Episode 578\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7571914943525898, Memory size: 9514 and Num Steps: 15\n",
      "Episode 579\tAverage Score: 0.01\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7564343028582372, Memory size: 9546 and Num Steps: 32\n",
      "Episode 580\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.755677868555379, Memory size: 9560 and Num Steps: 14\n",
      "Episode 581\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7549221906868236, Memory size: 9574 and Num Steps: 14\n",
      "Episode 582\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7541672684961368, Memory size: 9588 and Num Steps: 14\n",
      "Episode 583\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7534131012276407, Memory size: 9602 and Num Steps: 14\n",
      "Episode 584\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.752659688126413, Memory size: 9617 and Num Steps: 15\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 585\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7519070284382866, Memory size: 9631 and Num Steps: 14\n",
      "Episode 586\tAverage Score: 0.01\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7511551214098483, Memory size: 9645 and Num Steps: 14\n",
      "Episode 587\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7504039662884385, Memory size: 9679 and Num Steps: 34\n",
      "Episode 588\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7496535623221501, Memory size: 9731 and Num Steps: 52\n",
      "Episode 589\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7489039087598279, Memory size: 9764 and Num Steps: 33\n",
      "Episode 590\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7481550048510681, Memory size: 9779 and Num Steps: 15\n",
      "Episode 591\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.747406849846217, Memory size: 9831 and Num Steps: 52\n",
      "Episode 592\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7466594429963708, Memory size: 9848 and Num Steps: 17\n",
      "Episode 593\tAverage Score: 0.02\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.7459127835533744, Memory size: 9877 and Num Steps: 29\n",
      "Episode 594\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.745166870769821, Memory size: 9891 and Num Steps: 14\n",
      "Episode 595\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7444217038990512, Memory size: 9906 and Num Steps: 15\n",
      "Episode 596\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7436772821951522, Memory size: 9920 and Num Steps: 14\n",
      "Episode 597\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.742933604912957, Memory size: 9934 and Num Steps: 14\n",
      "Episode 598\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7421906713080441, Memory size: 9967 and Num Steps: 33\n",
      "Episode 599\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.741448480636736, Memory size: 9982 and Num Steps: 15\n",
      "Episode 600\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7407070321560992, Memory size: 9997 and Num Steps: 15\n",
      "Episode 601\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7399663251239431, Memory size: 10013 and Num Steps: 16\n",
      "Episode 602\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7392263587988193, Memory size: 10027 and Num Steps: 14\n",
      "Episode 603\tAverage Score: 0.02\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.7384871324400204, Memory size: 10102 and Num Steps: 75\n",
      "Episode 604\tAverage Score: 0.02\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7377486453075803, Memory size: 10119 and Num Steps: 17\n",
      "Episode 605\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7370108966622728, Memory size: 10152 and Num Steps: 33\n",
      "Episode 606\tAverage Score: 0.02\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.7362738857656105, Memory size: 10183 and Num Steps: 31\n",
      "Episode 607\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7355376118798449, Memory size: 10211 and Num Steps: 28\n",
      "Episode 608\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7348020742679651, Memory size: 10243 and Num Steps: 32\n",
      "Episode 609\tAverage Score: 0.02\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.734067272193697, Memory size: 10274 and Num Steps: 31\n",
      "Episode 610\tAverage Score: 0.02\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7333332049215034, Memory size: 10307 and Num Steps: 33\n",
      "Episode 611\tAverage Score: 0.03\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.7325998717165819, Memory size: 10338 and Num Steps: 31\n",
      "Episode 612\tAverage Score: 0.03\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7318672718448653, Memory size: 10369 and Num Steps: 31\n",
      "Episode 613\tAverage Score: 0.03\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.7311354045730205, Memory size: 10401 and Num Steps: 32\n",
      "Episode 614\tAverage Score: 0.03\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.7304042691684475, Memory size: 10431 and Num Steps: 30\n",
      "Episode 615\tAverage Score: 0.03\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.729673864899279, Memory size: 10462 and Num Steps: 31\n",
      "Episode 616\tAverage Score: 0.03\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7289441910343797, Memory size: 10493 and Num Steps: 31\n",
      "Episode 617\tAverage Score: 0.03\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7282152468433454, Memory size: 10509 and Num Steps: 16\n",
      "Episode 618\tAverage Score: 0.03\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.727487031596502, Memory size: 10561 and Num Steps: 52\n",
      "Episode 619\tAverage Score: 0.03\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7267595445649054, Memory size: 10574 and Num Steps: 13\n",
      "Episode 620\tAverage Score: 0.03\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.7260327850203405, Memory size: 10606 and Num Steps: 32\n",
      "Episode 621\tAverage Score: 0.03\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7253067522353203, Memory size: 10619 and Num Steps: 13\n",
      "Episode 622\tAverage Score: 0.03\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7245814454830849, Memory size: 10633 and Num Steps: 14\n",
      "Episode 623\tAverage Score: 0.03\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7238568640376019, Memory size: 10664 and Num Steps: 31\n",
      "Episode 624\tAverage Score: 0.03\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7231330071735642, Memory size: 10678 and Num Steps: 14\n",
      "Episode 625\tAverage Score: 0.03\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7224098741663907, Memory size: 10709 and Num Steps: 31\n",
      "Episode 626\tAverage Score: 0.03\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7216874642922243, Memory size: 10723 and Num Steps: 14\n",
      "Episode 627\tAverage Score: 0.03\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7209657768279321, Memory size: 10765 and Num Steps: 42\n",
      "Episode 628\tAverage Score: 0.03\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7202448110511042, Memory size: 10779 and Num Steps: 14\n",
      "Episode 629\tAverage Score: 0.03\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.719524566240053, Memory size: 10793 and Num Steps: 14\n",
      "Episode 630\tAverage Score: 0.03\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.718805041673813, Memory size: 10842 and Num Steps: 49\n",
      "Episode 631\tAverage Score: 0.03\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7180862366321391, Memory size: 10860 and Num Steps: 18\n",
      "Episode 632\tAverage Score: 0.03\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7173681503955069, Memory size: 10874 and Num Steps: 14\n",
      "Episode 633\tAverage Score: 0.03\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.7166507822451115, Memory size: 10905 and Num Steps: 31\n",
      "Episode 634\tAverage Score: 0.03\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7159341314628663, Memory size: 10919 and Num Steps: 14\n",
      "Episode 635\tAverage Score: 0.03\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7152181973314035, Memory size: 10969 and Num Steps: 50\n",
      "Episode 636\tAverage Score: 0.03\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7145029791340721, Memory size: 10983 and Num Steps: 14\n",
      "Episode 637\tAverage Score: 0.03\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.713788476154938, Memory size: 11014 and Num Steps: 31\n",
      "Episode 638\tAverage Score: 0.03\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7130746876787831, Memory size: 11028 and Num Steps: 14\n",
      "Episode 639\tAverage Score: 0.03\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.7123616129911043, Memory size: 11059 and Num Steps: 31\n",
      "Episode 640\tAverage Score: 0.03\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7116492513781132, Memory size: 11073 and Num Steps: 14\n",
      "Episode 641\tAverage Score: 0.03\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7109376021267351, Memory size: 11087 and Num Steps: 14\n",
      "Episode 642\tAverage Score: 0.03\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7102266645246084, Memory size: 11136 and Num Steps: 49\n",
      "Episode 643\tAverage Score: 0.03\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.7095164378600838, Memory size: 11166 and Num Steps: 30\n",
      "Episode 644\tAverage Score: 0.03\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7088069214222237, Memory size: 11192 and Num Steps: 26\n",
      "Episode 645\tAverage Score: 0.04\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.7080981145008014, Memory size: 11223 and Num Steps: 31\n",
      "Episode 646\tAverage Score: 0.04\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7073900163863006, Memory size: 11256 and Num Steps: 33\n",
      "Episode 647\tAverage Score: 0.04\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7066826263699143, Memory size: 11287 and Num Steps: 31\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 648\tAverage Score: 0.04\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.7059759437435444, Memory size: 11318 and Num Steps: 31\n",
      "Episode 649\tAverage Score: 0.04\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.7052699677998009, Memory size: 11350 and Num Steps: 32\n",
      "Episode 650\tAverage Score: 0.04\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.704564697832001, Memory size: 11403 and Num Steps: 53\n",
      "Episode 651\tAverage Score: 0.04\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7038601331341691, Memory size: 11436 and Num Steps: 33\n",
      "Episode 652\tAverage Score: 0.04\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7031562730010349, Memory size: 11465 and Num Steps: 29\n",
      "Episode 653\tAverage Score: 0.04\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7024531167280339, Memory size: 11498 and Num Steps: 33\n",
      "Episode 654\tAverage Score: 0.04\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7017506636113058, Memory size: 11531 and Num Steps: 33\n",
      "Episode 655\tAverage Score: 0.04\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.7010489129476946, Memory size: 11557 and Num Steps: 26\n",
      "Episode 656\tAverage Score: 0.04\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.7003478640347468, Memory size: 11606 and Num Steps: 49\n",
      "Episode 657\tAverage Score: 0.05\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6996475161707121, Memory size: 11680 and Num Steps: 74\n",
      "Episode 658\tAverage Score: 0.05\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6989478686545414, Memory size: 11710 and Num Steps: 30\n",
      "Episode 659\tAverage Score: 0.05\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6982489207858869, Memory size: 11741 and Num Steps: 31\n",
      "Episode 660\tAverage Score: 0.05\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6975506718651009, Memory size: 11767 and Num Steps: 26\n",
      "Episode 661\tAverage Score: 0.05\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6968531211932358, Memory size: 11798 and Num Steps: 31\n",
      "Episode 662\tAverage Score: 0.05\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6961562680720426, Memory size: 11831 and Num Steps: 33\n",
      "Episode 663\tAverage Score: 0.05\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6954601118039706, Memory size: 11861 and Num Steps: 30\n",
      "Episode 664\tAverage Score: 0.05\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6947646516921666, Memory size: 11875 and Num Steps: 14\n",
      "Episode 665\tAverage Score: 0.05\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6940698870404745, Memory size: 11908 and Num Steps: 33\n",
      "Episode 666\tAverage Score: 0.05\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.693375817153434, Memory size: 11922 and Num Steps: 14\n",
      "Episode 667\tAverage Score: 0.05\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6926824413362805, Memory size: 11954 and Num Steps: 32\n",
      "Episode 668\tAverage Score: 0.05\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6919897588949442, Memory size: 11984 and Num Steps: 30\n",
      "Episode 669\tAverage Score: 0.05\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6912977691360492, Memory size: 12034 and Num Steps: 50\n",
      "Episode 670\tAverage Score: 0.05\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6906064713669132, Memory size: 12047 and Num Steps: 13\n",
      "Episode 671\tAverage Score: 0.05\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6899158648955463, Memory size: 12078 and Num Steps: 31\n",
      "Episode 672\tAverage Score: 0.05\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.6892259490306508, Memory size: 12170 and Num Steps: 92\n",
      "Episode 673\tAverage Score: 0.05\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6885367230816202, Memory size: 12223 and Num Steps: 53\n",
      "Episode 674\tAverage Score: 0.05\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6878481863585385, Memory size: 12254 and Num Steps: 31\n",
      "Episode 675\tAverage Score: 0.05\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6871603381721799, Memory size: 12285 and Num Steps: 31\n",
      "Episode 676\tAverage Score: 0.05\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6864731778340077, Memory size: 12299 and Num Steps: 14\n",
      "Episode 677\tAverage Score: 0.05\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6857867046561738, Memory size: 12315 and Num Steps: 16\n",
      "Episode 678\tAverage Score: 0.06\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6851009179515176, Memory size: 12348 and Num Steps: 33\n",
      "Episode 679\tAverage Score: 0.06\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.6844158170335661, Memory size: 12440 and Num Steps: 92\n",
      "Episode 680\tAverage Score: 0.06\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6837314012165325, Memory size: 12471 and Num Steps: 31\n",
      "Episode 681\tAverage Score: 0.06\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.683047669815316, Memory size: 12502 and Num Steps: 31\n",
      "Episode 682\tAverage Score: 0.06\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6823646221455006, Memory size: 12533 and Num Steps: 31\n",
      "Episode 683\tAverage Score: 0.06\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6816822575233552, Memory size: 12550 and Num Steps: 17\n",
      "Episode 684\tAverage Score: 0.06\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.6810005752658318, Memory size: 12618 and Num Steps: 68\n",
      "Episode 685\tAverage Score: 0.06\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6803195746905659, Memory size: 12634 and Num Steps: 16\n",
      "Episode 686\tAverage Score: 0.06\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6796392551158754, Memory size: 12665 and Num Steps: 31\n",
      "Episode 687\tAverage Score: 0.06\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6789596158607595, Memory size: 12699 and Num Steps: 34\n",
      "Episode 688\tAverage Score: 0.06\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6782806562448987, Memory size: 12731 and Num Steps: 32\n",
      "Episode 689\tAverage Score: 0.06\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6776023755886539, Memory size: 12761 and Num Steps: 30\n",
      "Episode 690\tAverage Score: 0.06\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6769247732130652, Memory size: 12795 and Num Steps: 34\n",
      "Episode 691\tAverage Score: 0.06\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6762478484398522, Memory size: 12828 and Num Steps: 33\n",
      "Episode 692\tAverage Score: 0.06\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6755716005914123, Memory size: 12858 and Num Steps: 30\n",
      "Episode 693\tAverage Score: 0.06\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6748960289908209, Memory size: 12889 and Num Steps: 31\n",
      "Episode 694\tAverage Score: 0.07\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6742211329618301, Memory size: 12920 and Num Steps: 31\n",
      "Episode 695\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6735469118288683, Memory size: 12951 and Num Steps: 31\n",
      "Episode 696\tAverage Score: 0.07\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6728733649170394, Memory size: 12983 and Num Steps: 32\n",
      "Episode 697\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6722004915521224, Memory size: 13013 and Num Steps: 30\n",
      "Episode 698\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6715282910605702, Memory size: 13044 and Num Steps: 31\n",
      "Episode 699\tAverage Score: 0.07\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6708567627695097, Memory size: 13058 and Num Steps: 14\n",
      "Episode 700\tAverage Score: 0.07\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6701859060067401, Memory size: 13089 and Num Steps: 31\n",
      "Episode 701\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6695157201007333, Memory size: 13121 and Num Steps: 32\n",
      "Episode 702\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6688462043806327, Memory size: 13153 and Num Steps: 32\n",
      "Episode 703\tAverage Score: 0.07\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.668177358176252, Memory size: 13166 and Num Steps: 13\n",
      "Episode 704\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6675091808180758, Memory size: 13198 and Num Steps: 32\n",
      "Episode 705\tAverage Score: 0.07\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6668416716372577, Memory size: 13229 and Num Steps: 31\n",
      "Episode 706\tAverage Score: 0.07\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6661748299656204, Memory size: 13245 and Num Steps: 16\n",
      "Episode 707\tAverage Score: 0.07\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6655086551356548, Memory size: 13272 and Num Steps: 27\n",
      "Episode 708\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6648431464805191, Memory size: 13329 and Num Steps: 57\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 709\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6641783033340386, Memory size: 13364 and Num Steps: 35\n",
      "Episode 710\tAverage Score: 0.07\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6635141250307046, Memory size: 13394 and Num Steps: 30\n",
      "Episode 711\tAverage Score: 0.07\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6628506109056739, Memory size: 13425 and Num Steps: 31\n",
      "Episode 712\tAverage Score: 0.07\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6621877602947682, Memory size: 13455 and Num Steps: 30\n",
      "Episode 713\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6615255725344734, Memory size: 13508 and Num Steps: 53\n",
      "Episode 714\tAverage Score: 0.07\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.660864046961939, Memory size: 13538 and Num Steps: 30\n",
      "Episode 715\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.660203182914977, Memory size: 13557 and Num Steps: 19\n",
      "Episode 716\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6595429797320621, Memory size: 13592 and Num Steps: 35\n",
      "Episode 717\tAverage Score: 0.07\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.65888343675233, Memory size: 13609 and Num Steps: 17\n",
      "Episode 718\tAverage Score: 0.07\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6582245533155776, Memory size: 13639 and Num Steps: 30\n",
      "Episode 719\tAverage Score: 0.07\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6575663287622621, Memory size: 13653 and Num Steps: 14\n",
      "Episode 720\tAverage Score: 0.07\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6569087624334998, Memory size: 13667 and Num Steps: 14\n",
      "Episode 721\tAverage Score: 0.07\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6562518536710663, Memory size: 13681 and Num Steps: 14\n",
      "Episode 722\tAverage Score: 0.07\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6555956018173953, Memory size: 13695 and Num Steps: 14\n",
      "Episode 723\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6549400062155779, Memory size: 13727 and Num Steps: 32\n",
      "Episode 724\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6542850662093623, Memory size: 13759 and Num Steps: 32\n",
      "Episode 725\tAverage Score: 0.07\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6536307811431529, Memory size: 13790 and Num Steps: 31\n",
      "Episode 726\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6529771503620098, Memory size: 13823 and Num Steps: 33\n",
      "Episode 727\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6523241732116477, Memory size: 13856 and Num Steps: 33\n",
      "Episode 728\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6516718490384361, Memory size: 13888 and Num Steps: 32\n",
      "Episode 729\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6510201771893976, Memory size: 13941 and Num Steps: 53\n",
      "Episode 730\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6503691570122083, Memory size: 13974 and Num Steps: 33\n",
      "Episode 731\tAverage Score: 0.07\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6497187878551961, Memory size: 13988 and Num Steps: 14\n",
      "Episode 732\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6490690690673409, Memory size: 14018 and Num Steps: 30\n",
      "Episode 733\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6484199999982735, Memory size: 14051 and Num Steps: 33\n",
      "Episode 734\tAverage Score: 0.07\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6477715799982753, Memory size: 14082 and Num Steps: 31\n",
      "Episode 735\tAverage Score: 0.07\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.647123808418277, Memory size: 14096 and Num Steps: 14\n",
      "Episode 736\tAverage Score: 0.07\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6464766846098587, Memory size: 14110 and Num Steps: 14\n",
      "Episode 737\tAverage Score: 0.07\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6458302079252488, Memory size: 14140 and Num Steps: 30\n",
      "Episode 738\tAverage Score: 0.08\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.6451843777173236, Memory size: 14207 and Num Steps: 67\n",
      "Episode 739\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6445391933396063, Memory size: 14239 and Num Steps: 32\n",
      "Episode 740\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6438946541462667, Memory size: 14266 and Num Steps: 27\n",
      "Episode 741\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6432507594921204, Memory size: 14297 and Num Steps: 31\n",
      "Episode 742\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6426075087326283, Memory size: 14311 and Num Steps: 14\n",
      "Episode 743\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6419649012238956, Memory size: 14344 and Num Steps: 33\n",
      "Episode 744\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6413229363226718, Memory size: 14374 and Num Steps: 30\n",
      "Episode 745\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6406816133863491, Memory size: 14405 and Num Steps: 31\n",
      "Episode 746\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6400409317729627, Memory size: 14438 and Num Steps: 33\n",
      "Episode 747\tAverage Score: 0.08\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.6394008908411898, Memory size: 14504 and Num Steps: 66\n",
      "Episode 748\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6387614899503485, Memory size: 14540 and Num Steps: 36\n",
      "Episode 749\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6381227284603982, Memory size: 14574 and Num Steps: 34\n",
      "Episode 750\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6374846057319378, Memory size: 14605 and Num Steps: 31\n",
      "Episode 751\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6368471211262059, Memory size: 14636 and Num Steps: 31\n",
      "Episode 752\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6362102740050797, Memory size: 14666 and Num Steps: 30\n",
      "Episode 753\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6355740637310746, Memory size: 14698 and Num Steps: 32\n",
      "Episode 754\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6349384896673436, Memory size: 14731 and Num Steps: 33\n",
      "Episode 755\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6343035511776761, Memory size: 14746 and Num Steps: 15\n",
      "Episode 756\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6336692476264985, Memory size: 14777 and Num Steps: 31\n",
      "Episode 757\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.633035578378872, Memory size: 14792 and Num Steps: 15\n",
      "Episode 758\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6324025428004931, Memory size: 14843 and Num Steps: 51\n",
      "Episode 759\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6317701402576926, Memory size: 14876 and Num Steps: 33\n",
      "Episode 760\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6311383701174349, Memory size: 14906 and Num Steps: 30\n",
      "Episode 761\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6305072317473175, Memory size: 14920 and Num Steps: 14\n",
      "Episode 762\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6298767245155702, Memory size: 14934 and Num Steps: 14\n",
      "Episode 763\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6292468477910547, Memory size: 14965 and Num Steps: 31\n",
      "Episode 764\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6286176009432636, Memory size: 14995 and Num Steps: 30\n",
      "Episode 765\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6279889833423203, Memory size: 15026 and Num Steps: 31\n",
      "Episode 766\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.627360994358978, Memory size: 15057 and Num Steps: 31\n",
      "Episode 767\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.626733633364619, Memory size: 15087 and Num Steps: 30\n",
      "Episode 768\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6261068997312543, Memory size: 15116 and Num Steps: 29\n",
      "Episode 769\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6254807928315231, Memory size: 15148 and Num Steps: 32\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 770\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6248553120386916, Memory size: 15181 and Num Steps: 33\n",
      "Episode 771\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6242304567266529, Memory size: 15211 and Num Steps: 30\n",
      "Episode 772\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6236062262699262, Memory size: 15241 and Num Steps: 30\n",
      "Episode 773\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6229826200436563, Memory size: 15293 and Num Steps: 52\n",
      "Episode 774\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6223596374236127, Memory size: 15326 and Num Steps: 33\n",
      "Episode 775\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.621737277786189, Memory size: 15357 and Num Steps: 31\n",
      "Episode 776\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6211155405084029, Memory size: 15390 and Num Steps: 33\n",
      "Episode 777\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6204944249678944, Memory size: 15420 and Num Steps: 30\n",
      "Episode 778\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6198739305429266, Memory size: 15453 and Num Steps: 33\n",
      "Episode 779\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6192540566123836, Memory size: 15484 and Num Steps: 31\n",
      "Episode 780\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6186348025557713, Memory size: 15498 and Num Steps: 14\n",
      "Episode 781\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6180161677532154, Memory size: 15530 and Num Steps: 32\n",
      "Episode 782\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6173981515854623, Memory size: 15560 and Num Steps: 30\n",
      "Episode 783\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6167807534338768, Memory size: 15574 and Num Steps: 14\n",
      "Episode 784\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6161639726804429, Memory size: 15606 and Num Steps: 32\n",
      "Episode 785\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6155478087077625, Memory size: 15639 and Num Steps: 33\n",
      "Episode 786\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6149322608990547, Memory size: 15669 and Num Steps: 30\n",
      "Episode 787\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6143173286381557, Memory size: 15683 and Num Steps: 14\n",
      "Episode 788\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6137030113095175, Memory size: 15716 and Num Steps: 33\n",
      "Episode 789\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.613089308298208, Memory size: 15749 and Num Steps: 33\n",
      "Episode 790\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6124762189899098, Memory size: 15763 and Num Steps: 14\n",
      "Episode 791\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6118637427709198, Memory size: 15793 and Num Steps: 30\n",
      "Episode 792\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6112518790281489, Memory size: 15824 and Num Steps: 31\n",
      "Episode 793\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6106406271491208, Memory size: 15857 and Num Steps: 33\n",
      "Episode 794\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6100299865219717, Memory size: 15871 and Num Steps: 14\n",
      "Episode 795\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6094199565354497, Memory size: 15889 and Num Steps: 18\n",
      "Episode 796\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6088105365789143, Memory size: 15903 and Num Steps: 14\n",
      "Episode 797\tAverage Score: 0.07\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6082017260423354, Memory size: 15925 and Num Steps: 22\n",
      "Episode 798\tAverage Score: 0.07\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.607593524316293, Memory size: 15956 and Num Steps: 31\n",
      "Episode 799\tAverage Score: 0.07\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6069859307919767, Memory size: 15986 and Num Steps: 30\n",
      "Episode 800\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6063789448611847, Memory size: 16020 and Num Steps: 34\n",
      "Episode 801\tAverage Score: 0.07\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6057725659163236, Memory size: 16034 and Num Steps: 14\n",
      "Episode 802\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6051667933504072, Memory size: 16066 and Num Steps: 32\n",
      "Episode 803\tAverage Score: 0.07\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6045616265570568, Memory size: 16096 and Num Steps: 30\n",
      "Episode 804\tAverage Score: 0.07\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6039570649304997, Memory size: 16111 and Num Steps: 15\n",
      "Episode 805\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6033531078655693, Memory size: 16142 and Num Steps: 31\n",
      "Episode 806\tAverage Score: 0.07\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.6027497547577036, Memory size: 16161 and Num Steps: 19\n",
      "Episode 807\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6021470050029459, Memory size: 16194 and Num Steps: 33\n",
      "Episode 808\tAverage Score: 0.07\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.601544857997943, Memory size: 16224 and Num Steps: 30\n",
      "Episode 809\tAverage Score: 0.07\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.6009433131399451, Memory size: 16255 and Num Steps: 31\n",
      "Episode 810\tAverage Score: 0.07\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.6003423698268051, Memory size: 16305 and Num Steps: 50\n",
      "Episode 811\tAverage Score: 0.08\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.5997420274569784, Memory size: 16370 and Num Steps: 65\n",
      "Episode 812\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5991422854295214, Memory size: 16403 and Num Steps: 33\n",
      "Episode 813\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5985431431440918, Memory size: 16434 and Num Steps: 31\n",
      "Episode 814\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5979446000009477, Memory size: 16465 and Num Steps: 31\n",
      "Episode 815\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5973466554009468, Memory size: 16498 and Num Steps: 33\n",
      "Episode 816\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5967493087455459, Memory size: 16531 and Num Steps: 33\n",
      "Episode 817\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5961525594368003, Memory size: 16546 and Num Steps: 15\n",
      "Episode 818\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5955564068773636, Memory size: 16579 and Num Steps: 33\n",
      "Episode 819\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5949608504704862, Memory size: 16596 and Num Steps: 17\n",
      "Episode 820\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5943658896200157, Memory size: 16626 and Num Steps: 30\n",
      "Episode 821\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5937715237303957, Memory size: 16659 and Num Steps: 33\n",
      "Episode 822\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5931777522066652, Memory size: 16692 and Num Steps: 33\n",
      "Episode 823\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5925845744544586, Memory size: 16710 and Num Steps: 18\n",
      "Episode 824\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5919919898800041, Memory size: 16742 and Num Steps: 32\n",
      "Episode 825\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5913999978901241, Memory size: 16775 and Num Steps: 33\n",
      "Episode 826\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5908085978922339, Memory size: 16805 and Num Steps: 30\n",
      "Episode 827\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5902177892943418, Memory size: 16835 and Num Steps: 30\n",
      "Episode 828\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5896275715050474, Memory size: 16868 and Num Steps: 33\n",
      "Episode 829\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5890379439335424, Memory size: 16899 and Num Steps: 31\n",
      "Episode 830\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5884489059896089, Memory size: 16913 and Num Steps: 14\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 831\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5878604570836192, Memory size: 16966 and Num Steps: 53\n",
      "Episode 832\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5872725966265356, Memory size: 16995 and Num Steps: 29\n",
      "Episode 833\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.586685324029909, Memory size: 17043 and Num Steps: 48\n",
      "Episode 834\tAverage Score: 0.08\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.5860986387058792, Memory size: 17109 and Num Steps: 66\n",
      "Episode 835\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5855125400671732, Memory size: 17139 and Num Steps: 30\n",
      "Episode 836\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5849270275271061, Memory size: 17153 and Num Steps: 14\n",
      "Episode 837\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.584342100499579, Memory size: 17185 and Num Steps: 32\n",
      "Episode 838\tAverage Score: 0.08\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.5837577583990794, Memory size: 17295 and Num Steps: 110\n",
      "Episode 839\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5831740006406804, Memory size: 17325 and Num Steps: 30\n",
      "Episode 840\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5825908266400397, Memory size: 17358 and Num Steps: 33\n",
      "Episode 841\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5820082358133996, Memory size: 17372 and Num Steps: 14\n",
      "Episode 842\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5814262275775862, Memory size: 17405 and Num Steps: 33\n",
      "Episode 843\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5808448013500086, Memory size: 17438 and Num Steps: 33\n",
      "Episode 844\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5802639565486586, Memory size: 17468 and Num Steps: 30\n",
      "Episode 845\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.57968369259211, Memory size: 17497 and Num Steps: 29\n",
      "Episode 846\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5791040088995179, Memory size: 17527 and Num Steps: 30\n",
      "Episode 847\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5785249048906184, Memory size: 17560 and Num Steps: 33\n",
      "Episode 848\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5779463799857277, Memory size: 17574 and Num Steps: 14\n",
      "Episode 849\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.577368433605742, Memory size: 17604 and Num Steps: 30\n",
      "Episode 850\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5767910651721362, Memory size: 17618 and Num Steps: 14\n",
      "Episode 851\tAverage Score: 0.08\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5762142741069641, Memory size: 17632 and Num Steps: 14\n",
      "Episode 852\tAverage Score: 0.08\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.5756380598328571, Memory size: 17703 and Num Steps: 71\n",
      "Episode 853\tAverage Score: 0.08\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.5750624217730242, Memory size: 17772 and Num Steps: 69\n",
      "Episode 854\tAverage Score: 0.08\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5744873593512513, Memory size: 17805 and Num Steps: 33\n",
      "Episode 855\tAverage Score: 0.08\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.5739128719919, Memory size: 17874 and Num Steps: 69\n",
      "Episode 856\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5733389591199081, Memory size: 17904 and Num Steps: 30\n",
      "Episode 857\tAverage Score: 0.08\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.5727656201607881, Memory size: 17977 and Num Steps: 73\n",
      "Episode 858\tAverage Score: 0.08\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.5721928545406274, Memory size: 18047 and Num Steps: 70\n",
      "Episode 859\tAverage Score: 0.08\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.5716206616860868, Memory size: 18118 and Num Steps: 71\n",
      "Episode 860\tAverage Score: 0.08\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5710490410244007, Memory size: 18148 and Num Steps: 30\n",
      "Episode 861\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5704779919833762, Memory size: 18181 and Num Steps: 33\n",
      "Episode 862\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5699075139913929, Memory size: 18215 and Num Steps: 34\n",
      "Episode 863\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5693376064774015, Memory size: 18266 and Num Steps: 51\n",
      "Episode 864\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5687682688709241, Memory size: 18298 and Num Steps: 32\n",
      "Episode 865\tAverage Score: 0.09\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.5681995006020532, Memory size: 18368 and Num Steps: 70\n",
      "Episode 866\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5676313011014511, Memory size: 18400 and Num Steps: 32\n",
      "Episode 867\tAverage Score: 0.09\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5670636698003497, Memory size: 18430 and Num Steps: 30\n",
      "Episode 868\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5664966061305493, Memory size: 18482 and Num Steps: 52\n",
      "Episode 869\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5659301095244188, Memory size: 18508 and Num Steps: 26\n",
      "Episode 870\tAverage Score: 0.09\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5653641794148944, Memory size: 18523 and Num Steps: 15\n",
      "Episode 871\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5647988152354795, Memory size: 18572 and Num Steps: 49\n",
      "Episode 872\tAverage Score: 0.09\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.564234016420244, Memory size: 18585 and Num Steps: 13\n",
      "Episode 873\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5636697824038237, Memory size: 18637 and Num Steps: 52\n",
      "Episode 874\tAverage Score: 0.09\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.56310611262142, Memory size: 18651 and Num Steps: 14\n",
      "Episode 875\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5625430065087985, Memory size: 18683 and Num Steps: 32\n",
      "Episode 876\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5619804635022897, Memory size: 18716 and Num Steps: 33\n",
      "Episode 877\tAverage Score: 0.09\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.5614184830387874, Memory size: 18786 and Num Steps: 70\n",
      "Episode 878\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5608570645557486, Memory size: 18816 and Num Steps: 30\n",
      "Episode 879\tAverage Score: 0.09\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.5602962074911929, Memory size: 18886 and Num Steps: 70\n",
      "Episode 880\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5597359112837017, Memory size: 18919 and Num Steps: 33\n",
      "Episode 881\tAverage Score: 0.09\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.5591761753724179, Memory size: 18989 and Num Steps: 70\n",
      "Episode 882\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5586169991970455, Memory size: 19021 and Num Steps: 32\n",
      "Episode 883\tAverage Score: 0.09\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5580583821978485, Memory size: 19051 and Num Steps: 30\n",
      "Episode 884\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5575003238156506, Memory size: 19080 and Num Steps: 29\n",
      "Episode 885\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.556942823491835, Memory size: 19151 and Num Steps: 71\n",
      "Episode 886\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5563858806683432, Memory size: 19181 and Num Steps: 30\n",
      "Episode 887\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5558294947876748, Memory size: 19213 and Num Steps: 32\n",
      "Episode 888\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5552736652928871, Memory size: 19244 and Num Steps: 31\n",
      "Episode 889\tAverage Score: 0.09\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5547183916275943, Memory size: 19274 and Num Steps: 30\n",
      "Episode 890\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5541636732359667, Memory size: 19309 and Num Steps: 35\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 891\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5536095095627307, Memory size: 19339 and Num Steps: 30\n",
      "Episode 892\tAverage Score: 0.09\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.553055900053168, Memory size: 19370 and Num Steps: 31\n",
      "Episode 893\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5525028441531148, Memory size: 19403 and Num Steps: 33\n",
      "Episode 894\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5519503413089617, Memory size: 19436 and Num Steps: 33\n",
      "Episode 895\tAverage Score: 0.09\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5513983909676528, Memory size: 19469 and Num Steps: 33\n",
      "Episode 896\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5508469925766851, Memory size: 19501 and Num Steps: 32\n",
      "Episode 897\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5502961455841084, Memory size: 19532 and Num Steps: 31\n",
      "Episode 898\tAverage Score: 0.10\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.5497458494385243, Memory size: 19600 and Num Steps: 68\n",
      "Episode 899\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5491961035890858, Memory size: 19651 and Num Steps: 51\n",
      "Episode 900\tAverage Score: 0.10\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5486469074854967, Memory size: 19669 and Num Steps: 18\n",
      "Episode 901\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5480982605780111, Memory size: 19701 and Num Steps: 32\n",
      "Episode 902\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5475501623174331, Memory size: 19733 and Num Steps: 32\n",
      "Episode 903\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5470026121551157, Memory size: 19766 and Num Steps: 33\n",
      "Episode 904\tAverage Score: 0.10\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.5464556095429606, Memory size: 19833 and Num Steps: 67\n",
      "Episode 905\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5459091539334177, Memory size: 19863 and Num Steps: 30\n",
      "Episode 906\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5453632447794843, Memory size: 19893 and Num Steps: 30\n",
      "Episode 907\tAverage Score: 0.10\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5448178815347048, Memory size: 19923 and Num Steps: 30\n",
      "Episode 908\tAverage Score: 0.10\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5442730636531701, Memory size: 19939 and Num Steps: 16\n",
      "Episode 909\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5437287905895168, Memory size: 19991 and Num Steps: 52\n",
      "Episode 910\tAverage Score: 0.10\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.5431850617989273, Memory size: 20058 and Num Steps: 67\n",
      "Episode 911\tAverage Score: 0.10\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.5426418767371284, Memory size: 20125 and Num Steps: 67\n",
      "Episode 912\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5420992348603914, Memory size: 20158 and Num Steps: 33\n",
      "Episode 913\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5415571356255309, Memory size: 20189 and Num Steps: 31\n",
      "Episode 914\tAverage Score: 0.10\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.5410155784899053, Memory size: 20257 and Num Steps: 68\n",
      "Episode 915\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5404745629114155, Memory size: 20290 and Num Steps: 33\n",
      "Episode 916\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5399340883485041, Memory size: 20339 and Num Steps: 49\n",
      "Episode 917\tAverage Score: 0.10\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5393941542601556, Memory size: 20353 and Num Steps: 14\n",
      "Episode 918\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5388547601058954, Memory size: 20387 and Num Steps: 34\n",
      "Episode 919\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5383159053457895, Memory size: 20438 and Num Steps: 51\n",
      "Episode 920\tAverage Score: 0.10\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5377775894404437, Memory size: 20456 and Num Steps: 18\n",
      "Episode 921\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5372398118510032, Memory size: 20489 and Num Steps: 33\n",
      "Episode 922\tAverage Score: 0.10\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5367025720391523, Memory size: 20519 and Num Steps: 30\n",
      "Episode 923\tAverage Score: 0.10\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5361658694671131, Memory size: 20533 and Num Steps: 14\n",
      "Episode 924\tAverage Score: 0.10\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.535629703597646, Memory size: 20600 and Num Steps: 67\n",
      "Episode 925\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5350940738940484, Memory size: 20635 and Num Steps: 35\n",
      "Episode 926\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5345589798201543, Memory size: 20668 and Num Steps: 33\n",
      "Episode 927\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5340244208403342, Memory size: 20699 and Num Steps: 31\n",
      "Episode 928\tAverage Score: 0.10\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5334903964194938, Memory size: 20729 and Num Steps: 30\n",
      "Episode 929\tAverage Score: 0.10\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.5329569060230743, Memory size: 20795 and Num Steps: 66\n",
      "Episode 930\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5324239491170513, Memory size: 20829 and Num Steps: 34\n",
      "Episode 931\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5318915251679341, Memory size: 20859 and Num Steps: 30\n",
      "Episode 932\tAverage Score: 0.10\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5313596336427663, Memory size: 20873 and Num Steps: 14\n",
      "Episode 933\tAverage Score: 0.10\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.5308282740091235, Memory size: 20942 and Num Steps: 69\n",
      "Episode 934\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5302974457351144, Memory size: 20975 and Num Steps: 33\n",
      "Episode 935\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5297671482893792, Memory size: 21007 and Num Steps: 32\n",
      "Episode 936\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5292373811410899, Memory size: 21040 and Num Steps: 33\n",
      "Episode 937\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5287081437599488, Memory size: 21073 and Num Steps: 33\n",
      "Episode 938\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5281794356161889, Memory size: 21104 and Num Steps: 31\n",
      "Episode 939\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5276512561805726, Memory size: 21135 and Num Steps: 31\n",
      "Episode 940\tAverage Score: 0.10\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.527123604924392, Memory size: 21204 and Num Steps: 69\n",
      "Episode 941\tAverage Score: 0.11\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5265964813194677, Memory size: 21234 and Num Steps: 30\n",
      "Episode 942\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5260698848381482, Memory size: 21268 and Num Steps: 34\n",
      "Episode 943\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.52554381495331, Memory size: 21298 and Num Steps: 30\n",
      "Episode 944\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5250182711383568, Memory size: 21328 and Num Steps: 30\n",
      "Episode 945\tAverage Score: 0.11\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.5244932528672184, Memory size: 21393 and Num Steps: 65\n",
      "Episode 946\tAverage Score: 0.11\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5239687596143512, Memory size: 21424 and Num Steps: 31\n",
      "Episode 947\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5234447908547368, Memory size: 21456 and Num Steps: 32\n",
      "Episode 948\tAverage Score: 0.11\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.5229213460638821, Memory size: 21522 and Num Steps: 66\n",
      "Episode 949\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5223984247178182, Memory size: 21548 and Num Steps: 26\n",
      "Episode 950\tAverage Score: 0.11\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.5218760262931004, Memory size: 21637 and Num Steps: 89\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 951\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5213541502668073, Memory size: 21670 and Num Steps: 33\n",
      "Episode 952\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5208327961165404, Memory size: 21702 and Num Steps: 32\n",
      "Episode 953\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.520311963320424, Memory size: 21753 and Num Steps: 51\n",
      "Episode 954\tAverage Score: 0.11\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5197916513571035, Memory size: 21783 and Num Steps: 30\n",
      "Episode 955\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5192718597057464, Memory size: 21817 and Num Steps: 34\n",
      "Episode 956\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5187525878460406, Memory size: 21847 and Num Steps: 30\n",
      "Episode 957\tAverage Score: 0.11\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5182338352581947, Memory size: 21861 and Num Steps: 14\n",
      "Episode 958\tAverage Score: 0.10\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5177156014229364, Memory size: 21891 and Num Steps: 30\n",
      "Episode 959\tAverage Score: 0.10\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5171978858215135, Memory size: 21921 and Num Steps: 30\n",
      "Episode 960\tAverage Score: 0.10\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.516680687935692, Memory size: 22008 and Num Steps: 87\n",
      "Episode 961\tAverage Score: 0.10\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5161640072477562, Memory size: 22025 and Num Steps: 17\n",
      "Episode 962\tAverage Score: 0.10\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5156478432405085, Memory size: 22041 and Num Steps: 16\n",
      "Episode 963\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5151321953972681, Memory size: 22074 and Num Steps: 33\n",
      "Episode 964\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5146170632018707, Memory size: 22105 and Num Steps: 31\n",
      "Episode 965\tAverage Score: 0.10\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.5141024461386688, Memory size: 22126 and Num Steps: 21\n",
      "Episode 966\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5135883436925303, Memory size: 22178 and Num Steps: 52\n",
      "Episode 967\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5130747553488376, Memory size: 22209 and Num Steps: 31\n",
      "Episode 968\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5125616805934888, Memory size: 22240 and Num Steps: 31\n",
      "Episode 969\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5120491189128954, Memory size: 22292 and Num Steps: 52\n",
      "Episode 970\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5115370697939825, Memory size: 22325 and Num Steps: 33\n",
      "Episode 971\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5110255327241885, Memory size: 22373 and Num Steps: 48\n",
      "Episode 972\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5105145071914643, Memory size: 22427 and Num Steps: 54\n",
      "Episode 973\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5100039926842728, Memory size: 22460 and Num Steps: 33\n",
      "Episode 974\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5094939886915886, Memory size: 22493 and Num Steps: 33\n",
      "Episode 975\tAverage Score: 0.10\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.508984494702897, Memory size: 22564 and Num Steps: 71\n",
      "Episode 976\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.508475510208194, Memory size: 22597 and Num Steps: 33\n",
      "Episode 977\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5079670346979859, Memory size: 22630 and Num Steps: 33\n",
      "Episode 978\tAverage Score: 0.10\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.5074590676632879, Memory size: 22660 and Num Steps: 30\n",
      "Episode 979\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5069516085956246, Memory size: 22713 and Num Steps: 53\n",
      "Episode 980\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.506444656987029, Memory size: 22745 and Num Steps: 32\n",
      "Episode 981\tAverage Score: 0.10\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.5059382123300419, Memory size: 22815 and Num Steps: 70\n",
      "Episode 982\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5054322741177119, Memory size: 22846 and Num Steps: 31\n",
      "Episode 983\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5049268418435942, Memory size: 22911 and Num Steps: 65\n",
      "Episode 984\tAverage Score: 0.10\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.5044219150017506, Memory size: 23000 and Num Steps: 89\n",
      "Episode 985\tAverage Score: 0.10\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5039174930867488, Memory size: 23031 and Num Steps: 31\n",
      "Episode 986\tAverage Score: 0.11\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.5034135755936621, Memory size: 23179 and Num Steps: 148\n",
      "Episode 987\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5029101620180684, Memory size: 23250 and Num Steps: 71\n",
      "Episode 988\tAverage Score: 0.11\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.5024072518560504, Memory size: 23318 and Num Steps: 68\n",
      "Episode 989\tAverage Score: 0.11\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.5019048446041943, Memory size: 23391 and Num Steps: 73\n",
      "Episode 990\tAverage Score: 0.11\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.50140293975959, Memory size: 23405 and Num Steps: 14\n",
      "Episode 991\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.5009015368198305, Memory size: 23457 and Num Steps: 52\n",
      "Episode 992\tAverage Score: 0.11\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.5004006352830107, Memory size: 23531 and Num Steps: 74\n",
      "Episode 993\tAverage Score: 0.11\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.49990023464772765, Memory size: 23544 and Num Steps: 13\n",
      "Episode 994\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.49940033441307996, Memory size: 23576 and Num Steps: 32\n",
      "Episode 995\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.49890093407866687, Memory size: 23607 and Num Steps: 31\n",
      "Episode 996\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4984020331445882, Memory size: 23638 and Num Steps: 31\n",
      "Episode 997\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4979036311114436, Memory size: 23686 and Num Steps: 48\n",
      "Episode 998\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4974057274803322, Memory size: 23775 and Num Steps: 89\n",
      "Episode 999\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4969083217528518, Memory size: 23805 and Num Steps: 30\n",
      "Episode 1000\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.49641141343109896, Memory size: 23857 and Num Steps: 52\n",
      "Saved Model: Episode 1000\tAverage Score: 0.11\n",
      "Episode 1001\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.49591500201766786, Memory size: 23890 and Num Steps: 33\n",
      "Episode 1002\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4954190870156502, Memory size: 23925 and Num Steps: 35\n",
      "Episode 1003\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.49492366792863457, Memory size: 23955 and Num Steps: 30\n",
      "Episode 1004\tAverage Score: 0.11\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.49442874426070593, Memory size: 24044 and Num Steps: 89\n",
      "Episode 1005\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.49393431551644523, Memory size: 24074 and Num Steps: 30\n",
      "Episode 1006\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.49344038120092876, Memory size: 24105 and Num Steps: 31\n",
      "Episode 1007\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.49294694081972784, Memory size: 24156 and Num Steps: 51\n",
      "Episode 1008\tAverage Score: 0.11\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.4924539938789081, Memory size: 24224 and Num Steps: 68\n",
      "Episode 1009\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4919615398850292, Memory size: 24255 and Num Steps: 31\n",
      "Episode 1010\tAverage Score: 0.11\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.49146957834514415, Memory size: 24324 and Num Steps: 69\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1011\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.49097810876679904, Memory size: 24364 and Num Steps: 40\n",
      "Episode 1012\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.49048713065803223, Memory size: 24415 and Num Steps: 51\n",
      "Episode 1013\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4899966435273742, Memory size: 24460 and Num Steps: 45\n",
      "Episode 1014\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4895066468838468, Memory size: 24512 and Num Steps: 52\n",
      "Episode 1015\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.48901714023696297, Memory size: 24564 and Num Steps: 52\n",
      "Episode 1016\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.488528123096726, Memory size: 24597 and Num Steps: 33\n",
      "Episode 1017\tAverage Score: 0.11\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.4880395949736293, Memory size: 24666 and Num Steps: 69\n",
      "Episode 1018\tAverage Score: 0.11\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.48755155537865563, Memory size: 24698 and Num Steps: 32\n",
      "Episode 1019\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.487064003823277, Memory size: 24747 and Num Steps: 49\n",
      "Episode 1020\tAverage Score: 0.11\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.4865769398194537, Memory size: 24761 and Num Steps: 14\n",
      "Episode 1021\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.48609036287963425, Memory size: 24813 and Num Steps: 52\n",
      "Episode 1022\tAverage Score: 0.11\tCurrent Score: 0.2900000046938658\n",
      "Noise Scaling: 0.48560427251675464, Memory size: 24918 and Num Steps: 105\n",
      "Episode 1023\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4851186682442379, Memory size: 24951 and Num Steps: 33\n",
      "Episode 1024\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.48463354957599364, Memory size: 25000 and Num Steps: 49\n",
      "Episode 1025\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.48414891602641763, Memory size: 25032 and Num Steps: 32\n",
      "Episode 1026\tAverage Score: 0.11\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.48366476711039125, Memory size: 25046 and Num Steps: 14\n",
      "Episode 1027\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4831811023432808, Memory size: 25077 and Num Steps: 31\n",
      "Episode 1028\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4826979212409376, Memory size: 25109 and Num Steps: 32\n",
      "Episode 1029\tAverage Score: 0.11\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.48221522331969663, Memory size: 25191 and Num Steps: 82\n",
      "Episode 1030\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4817330080963769, Memory size: 25225 and Num Steps: 34\n",
      "Episode 1031\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.48125127508828053, Memory size: 25276 and Num Steps: 51\n",
      "Episode 1032\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4807700238131923, Memory size: 25347 and Num Steps: 71\n",
      "Episode 1033\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4802892537893791, Memory size: 25377 and Num Steps: 30\n",
      "Episode 1034\tAverage Score: 0.11\tCurrent Score: 0.2900000046938658\n",
      "Noise Scaling: 0.4798089645355897, Memory size: 25484 and Num Steps: 107\n",
      "Episode 1035\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4793291555710541, Memory size: 25515 and Num Steps: 31\n",
      "Episode 1036\tAverage Score: 0.12\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.47884982641548307, Memory size: 25626 and Num Steps: 111\n",
      "Episode 1037\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4783709765890676, Memory size: 25657 and Num Steps: 31\n",
      "Episode 1038\tAverage Score: 0.12\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.4778926056124785, Memory size: 25806 and Num Steps: 149\n",
      "Episode 1039\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.477414713006866, Memory size: 25857 and Num Steps: 51\n",
      "Episode 1040\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.47693729829385917, Memory size: 25871 and Num Steps: 14\n",
      "Episode 1041\tAverage Score: 0.12\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.47646036099556527, Memory size: 26001 and Num Steps: 130\n",
      "Episode 1042\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.47598390063456975, Memory size: 26033 and Num Steps: 32\n",
      "Episode 1043\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.47550791673393517, Memory size: 26047 and Num Steps: 14\n",
      "Episode 1044\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4750324088172012, Memory size: 26083 and Num Steps: 36\n",
      "Episode 1045\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.474557376408384, Memory size: 26136 and Num Steps: 53\n",
      "Episode 1046\tAverage Score: 0.12\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.47408281903197563, Memory size: 26255 and Num Steps: 119\n",
      "Episode 1047\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.47360873621294364, Memory size: 26286 and Num Steps: 31\n",
      "Episode 1048\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.4731351274767307, Memory size: 26317 and Num Steps: 31\n",
      "Episode 1049\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.472661992349254, Memory size: 26331 and Num Steps: 14\n",
      "Episode 1050\tAverage Score: 0.11\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.47218933035690475, Memory size: 26374 and Num Steps: 43\n",
      "Episode 1051\tAverage Score: 0.12\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.4717171410265478, Memory size: 26564 and Num Steps: 190\n",
      "Episode 1052\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.47124542388552126, Memory size: 26578 and Num Steps: 14\n",
      "Episode 1053\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4707741784616358, Memory size: 26609 and Num Steps: 31\n",
      "Episode 1054\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.47030340428317413, Memory size: 26639 and Num Steps: 30\n",
      "Episode 1055\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.46983310087889096, Memory size: 26653 and Num Steps: 14\n",
      "Episode 1056\tAverage Score: 0.12\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.46936326777801207, Memory size: 26684 and Num Steps: 31\n",
      "Episode 1057\tAverage Score: 0.12\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.46889390451023405, Memory size: 26753 and Num Steps: 69\n",
      "Episode 1058\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4684250106057238, Memory size: 26783 and Num Steps: 30\n",
      "Episode 1059\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4679565855951181, Memory size: 26815 and Num Steps: 32\n",
      "Episode 1060\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.467488629009523, Memory size: 26829 and Num Steps: 14\n",
      "Episode 1061\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.46702114038051346, Memory size: 26859 and Num Steps: 30\n",
      "Episode 1062\tAverage Score: 0.12\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.4665541192401329, Memory size: 26969 and Num Steps: 110\n",
      "Episode 1063\tAverage Score: 0.12\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.4660875651208928, Memory size: 27058 and Num Steps: 89\n",
      "Episode 1064\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4656214775557719, Memory size: 27107 and Num Steps: 49\n",
      "Episode 1065\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.46515585607821613, Memory size: 27138 and Num Steps: 31\n",
      "Episode 1066\tAverage Score: 0.12\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.46469070022213793, Memory size: 27206 and Num Steps: 68\n",
      "Episode 1067\tAverage Score: 0.12\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.4642260095219158, Memory size: 27277 and Num Steps: 71\n",
      "Episode 1068\tAverage Score: 0.12\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.46376178351239383, Memory size: 27348 and Num Steps: 71\n",
      "Episode 1069\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.46329802172888146, Memory size: 27420 and Num Steps: 72\n",
      "Episode 1070\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.46283472370715256, Memory size: 27449 and Num Steps: 29\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1071\tAverage Score: 0.12\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.46237188898344544, Memory size: 27479 and Num Steps: 30\n",
      "Episode 1072\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.461909517094462, Memory size: 27532 and Num Steps: 53\n",
      "Episode 1073\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4614476075773675, Memory size: 27580 and Num Steps: 48\n",
      "Episode 1074\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.46098615996979014, Memory size: 27610 and Num Steps: 30\n",
      "Episode 1075\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.46052517380982033, Memory size: 27640 and Num Steps: 30\n",
      "Episode 1076\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.46006464863601054, Memory size: 27655 and Num Steps: 15\n",
      "Episode 1077\tAverage Score: 0.12\tCurrent Score: 0.2900000046938658\n",
      "Noise Scaling: 0.4596045839873745, Memory size: 27760 and Num Steps: 105\n",
      "Episode 1078\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.45914497940338717, Memory size: 27790 and Num Steps: 30\n",
      "Episode 1079\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.45868583442398375, Memory size: 27822 and Num Steps: 32\n",
      "Episode 1080\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.45822714858955976, Memory size: 27852 and Num Steps: 30\n",
      "Episode 1081\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4577689214409702, Memory size: 27901 and Num Steps: 49\n",
      "Episode 1082\tAverage Score: 0.13\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.45731115251952925, Memory size: 28088 and Num Steps: 187\n",
      "Episode 1083\tAverage Score: 0.13\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.45685384136700974, Memory size: 28193 and Num Steps: 105\n",
      "Episode 1084\tAverage Score: 0.13\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.45639698752564273, Memory size: 28255 and Num Steps: 62\n",
      "Episode 1085\tAverage Score: 0.13\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.4559405905381171, Memory size: 28269 and Num Steps: 14\n",
      "Episode 1086\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4554846499475789, Memory size: 28321 and Num Steps: 52\n",
      "Episode 1087\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4550291652976314, Memory size: 28378 and Num Steps: 57\n",
      "Episode 1088\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4545741361323337, Memory size: 28410 and Num Steps: 32\n",
      "Episode 1089\tAverage Score: 0.12\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.4541195619962014, Memory size: 28478 and Num Steps: 68\n",
      "Episode 1090\tAverage Score: 0.13\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.4536654424342052, Memory size: 28509 and Num Steps: 31\n",
      "Episode 1091\tAverage Score: 0.13\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.453211776991771, Memory size: 28540 and Num Steps: 31\n",
      "Episode 1092\tAverage Score: 0.12\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.4527585652147792, Memory size: 28571 and Num Steps: 31\n",
      "Episode 1093\tAverage Score: 0.13\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.45230580664956443, Memory size: 28643 and Num Steps: 72\n",
      "Episode 1094\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.45185350084291487, Memory size: 28695 and Num Steps: 52\n",
      "Episode 1095\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.45140164734207194, Memory size: 28726 and Num Steps: 31\n",
      "Episode 1096\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4509502456947299, Memory size: 28758 and Num Steps: 32\n",
      "Episode 1097\tAverage Score: 0.13\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.45049929544903516, Memory size: 28788 and Num Steps: 30\n",
      "Episode 1098\tAverage Score: 0.13\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.4500487961535861, Memory size: 28854 and Num Steps: 66\n",
      "Episode 1099\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.44959874735743255, Memory size: 28907 and Num Steps: 53\n",
      "Episode 1100\tAverage Score: 0.13\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.4491491486100751, Memory size: 28921 and Num Steps: 14\n",
      "Episode 1101\tAverage Score: 0.13\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.44869999946146505, Memory size: 29022 and Num Steps: 101\n",
      "Episode 1102\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.44825129946200354, Memory size: 29053 and Num Steps: 31\n",
      "Episode 1103\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4478030481625416, Memory size: 29085 and Num Steps: 32\n",
      "Episode 1104\tAverage Score: 0.13\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.447355245114379, Memory size: 29099 and Num Steps: 14\n",
      "Episode 1105\tAverage Score: 0.13\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.44690788986926466, Memory size: 29207 and Num Steps: 108\n",
      "Episode 1106\tAverage Score: 0.13\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.4464609819793954, Memory size: 29221 and Num Steps: 14\n",
      "Episode 1107\tAverage Score: 0.13\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.44601452099741595, Memory size: 29235 and Num Steps: 14\n",
      "Episode 1108\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.44556850647641855, Memory size: 29278 and Num Steps: 43\n",
      "Episode 1109\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.4451229379699421, Memory size: 29295 and Num Steps: 17\n",
      "Episode 1110\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.4446778150319722, Memory size: 29328 and Num Steps: 33\n",
      "Episode 1111\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.44423313721694024, Memory size: 29376 and Num Steps: 48\n",
      "Episode 1112\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.44378890407972327, Memory size: 29400 and Num Steps: 24\n",
      "Episode 1113\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.44334511517564357, Memory size: 29452 and Num Steps: 52\n",
      "Episode 1114\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.44290177006046794, Memory size: 29482 and Num Steps: 30\n",
      "Episode 1115\tAverage Score: 0.12\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.4424588682904074, Memory size: 29553 and Num Steps: 71\n",
      "Episode 1116\tAverage Score: 0.12\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.44201640942211706, Memory size: 29583 and Num Steps: 30\n",
      "Episode 1117\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4415743930126949, Memory size: 29634 and Num Steps: 51\n",
      "Episode 1118\tAverage Score: 0.12\tCurrent Score: 0.09000000357627869\n",
      "Noise Scaling: 0.44113281861968223, Memory size: 29653 and Num Steps: 19\n",
      "Episode 1119\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.44069168580106255, Memory size: 29667 and Num Steps: 14\n",
      "Episode 1120\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4402509941152615, Memory size: 29718 and Num Steps: 51\n",
      "Episode 1121\tAverage Score: 0.12\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.4398107431211462, Memory size: 29749 and Num Steps: 31\n",
      "Episode 1122\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4393709323780251, Memory size: 29802 and Num Steps: 53\n",
      "Episode 1123\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.43893156144564704, Memory size: 29835 and Num Steps: 33\n",
      "Episode 1124\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4384926298842014, Memory size: 29906 and Num Steps: 71\n",
      "Episode 1125\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4380541372543172, Memory size: 29935 and Num Steps: 29\n",
      "Episode 1126\tAverage Score: 0.12\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.4376160831170629, Memory size: 30042 and Num Steps: 107\n",
      "Episode 1127\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4371784670339458, Memory size: 30089 and Num Steps: 47\n",
      "Episode 1128\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.43674128856691186, Memory size: 30103 and Num Steps: 14\n",
      "Episode 1129\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.43630454727834495, Memory size: 30152 and Num Steps: 49\n",
      "Episode 1130\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4358682427310666, Memory size: 30186 and Num Steps: 34\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1131\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.43543237448833555, Memory size: 30223 and Num Steps: 37\n",
      "Episode 1132\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.4349969421138472, Memory size: 30241 and Num Steps: 18\n",
      "Episode 1133\tAverage Score: 0.12\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.43456194517173335, Memory size: 30350 and Num Steps: 109\n",
      "Episode 1134\tAverage Score: 0.12\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.43412738322656164, Memory size: 30422 and Num Steps: 72\n",
      "Episode 1135\tAverage Score: 0.12\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.43369325584333507, Memory size: 30557 and Num Steps: 135\n",
      "Episode 1136\tAverage Score: 0.12\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.4332595625874917, Memory size: 30588 and Num Steps: 31\n",
      "Episode 1137\tAverage Score: 0.12\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.43282630302490427, Memory size: 30659 and Num Steps: 71\n",
      "Episode 1138\tAverage Score: 0.12\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.43239347672187933, Memory size: 30795 and Num Steps: 136\n",
      "Episode 1139\tAverage Score: 0.12\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.4319610832451575, Memory size: 30899 and Num Steps: 104\n",
      "Episode 1140\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4315291221619123, Memory size: 30931 and Num Steps: 32\n",
      "Episode 1141\tAverage Score: 0.13\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.43109759303975037, Memory size: 31155 and Num Steps: 224\n",
      "Episode 1142\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.43066649544671065, Memory size: 31185 and Num Steps: 30\n",
      "Episode 1143\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.43023582895126394, Memory size: 31214 and Num Steps: 29\n",
      "Episode 1144\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.42980559312231265, Memory size: 31247 and Num Steps: 33\n",
      "Episode 1145\tAverage Score: 0.13\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.42937578752919037, Memory size: 31261 and Num Steps: 14\n",
      "Episode 1146\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.42894641174166115, Memory size: 31312 and Num Steps: 51\n",
      "Episode 1147\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4285174653299195, Memory size: 31349 and Num Steps: 37\n",
      "Episode 1148\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.42808894786458956, Memory size: 31416 and Num Steps: 67\n",
      "Episode 1149\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.427660858916725, Memory size: 31449 and Num Steps: 33\n",
      "Episode 1150\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.42723319805780824, Memory size: 31500 and Num Steps: 51\n",
      "Episode 1151\tAverage Score: 0.12\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.4268059648597505, Memory size: 31598 and Num Steps: 98\n",
      "Episode 1152\tAverage Score: 0.12\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.4263791588948907, Memory size: 31628 and Num Steps: 30\n",
      "Episode 1153\tAverage Score: 0.13\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.4259527797359958, Memory size: 31739 and Num Steps: 111\n",
      "Episode 1154\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.4255268269562598, Memory size: 31753 and Num Steps: 14\n",
      "Episode 1155\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4251013001293036, Memory size: 31784 and Num Steps: 31\n",
      "Episode 1156\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.42467619882917423, Memory size: 31801 and Num Steps: 17\n",
      "Episode 1157\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.4242515226303451, Memory size: 31815 and Num Steps: 14\n",
      "Episode 1158\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.42382727110771473, Memory size: 31864 and Num Steps: 49\n",
      "Episode 1159\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.423403443836607, Memory size: 31880 and Num Steps: 16\n",
      "Episode 1160\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.42298004039277043, Memory size: 31932 and Num Steps: 52\n",
      "Episode 1161\tAverage Score: 0.12\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.4225570603523776, Memory size: 32039 and Num Steps: 107\n",
      "Episode 1162\tAverage Score: 0.12\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.4221345032920253, Memory size: 32110 and Num Steps: 71\n",
      "Episode 1163\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.42171236878873325, Memory size: 32177 and Num Steps: 67\n",
      "Episode 1164\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4212906564199445, Memory size: 32244 and Num Steps: 67\n",
      "Episode 1165\tAverage Score: 0.12\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.42086936576352457, Memory size: 32334 and Num Steps: 90\n",
      "Episode 1166\tAverage Score: 0.13\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.42044849639776105, Memory size: 32481 and Num Steps: 147\n",
      "Episode 1167\tAverage Score: 0.13\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.4200280479013633, Memory size: 32609 and Num Steps: 128\n",
      "Episode 1168\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.4196080198534619, Memory size: 32623 and Num Steps: 14\n",
      "Episode 1169\tAverage Score: 0.13\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.41918841183360844, Memory size: 32710 and Num Steps: 87\n",
      "Episode 1170\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.41876922342177486, Memory size: 32766 and Num Steps: 56\n",
      "Episode 1171\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.41835045419835304, Memory size: 32795 and Num Steps: 29\n",
      "Episode 1172\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4179321037441547, Memory size: 32827 and Num Steps: 32\n",
      "Episode 1173\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.41751417164041055, Memory size: 32857 and Num Steps: 30\n",
      "Episode 1174\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4170966574687701, Memory size: 32884 and Num Steps: 27\n",
      "Episode 1175\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.41667956081130136, Memory size: 32935 and Num Steps: 51\n",
      "Episode 1176\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.41626288125049005, Memory size: 32966 and Num Steps: 31\n",
      "Episode 1177\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4158466183692396, Memory size: 33016 and Num Steps: 50\n",
      "Episode 1178\tAverage Score: 0.13\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.41543077175087034, Memory size: 33108 and Num Steps: 92\n",
      "Episode 1179\tAverage Score: 0.13\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.41501534097911946, Memory size: 33200 and Num Steps: 92\n",
      "Episode 1180\tAverage Score: 0.13\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.41460032563814037, Memory size: 33230 and Num Steps: 30\n",
      "Episode 1181\tAverage Score: 0.13\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.4141857253125022, Memory size: 33244 and Num Steps: 14\n",
      "Episode 1182\tAverage Score: 0.12\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.4137715395871897, Memory size: 33276 and Num Steps: 32\n",
      "Episode 1183\tAverage Score: 0.12\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.4133577680476025, Memory size: 33369 and Num Steps: 93\n",
      "Episode 1184\tAverage Score: 0.12\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.4129444102795549, Memory size: 33490 and Num Steps: 121\n",
      "Episode 1185\tAverage Score: 0.12\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.41253146586927536, Memory size: 33597 and Num Steps: 107\n",
      "Episode 1186\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4121189344034061, Memory size: 33651 and Num Steps: 54\n",
      "Episode 1187\tAverage Score: 0.12\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.4117068154690027, Memory size: 33715 and Num Steps: 64\n",
      "Episode 1188\tAverage Score: 0.13\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.4112951086535337, Memory size: 33804 and Num Steps: 89\n",
      "Episode 1189\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.4108838135448801, Memory size: 33818 and Num Steps: 14\n",
      "Episode 1190\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.41047292973133526, Memory size: 33870 and Num Steps: 52\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1191\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.4100624568016039, Memory size: 33890 and Num Steps: 20\n",
      "Episode 1192\tAverage Score: 0.13\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.40965239434480233, Memory size: 34016 and Num Steps: 126\n",
      "Episode 1193\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4092427419504575, Memory size: 34046 and Num Steps: 30\n",
      "Episode 1194\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.40883349920850703, Memory size: 34060 and Num Steps: 14\n",
      "Episode 1195\tAverage Score: 0.12\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.40842466570929853, Memory size: 34150 and Num Steps: 90\n",
      "Episode 1196\tAverage Score: 0.13\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.40801624104358925, Memory size: 34244 and Num Steps: 94\n",
      "Episode 1197\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.40760822480254566, Memory size: 34275 and Num Steps: 31\n",
      "Episode 1198\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4072006165777431, Memory size: 34306 and Num Steps: 31\n",
      "Episode 1199\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.40679341596116536, Memory size: 34324 and Num Steps: 18\n",
      "Episode 1200\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4063866225452042, Memory size: 34375 and Num Steps: 51\n",
      "Episode 1201\tAverage Score: 0.12\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.405980235922659, Memory size: 34446 and Num Steps: 71\n",
      "Episode 1202\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.40557425568673633, Memory size: 34460 and Num Steps: 14\n",
      "Episode 1203\tAverage Score: 0.12\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.4051686814310496, Memory size: 34551 and Num Steps: 91\n",
      "Episode 1204\tAverage Score: 0.12\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.4047635127496186, Memory size: 34565 and Num Steps: 14\n",
      "Episode 1205\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.40435874923686893, Memory size: 34636 and Num Steps: 71\n",
      "Episode 1206\tAverage Score: 0.12\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.40395439048763204, Memory size: 34689 and Num Steps: 53\n",
      "Episode 1207\tAverage Score: 0.13\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.4035504360971444, Memory size: 34759 and Num Steps: 70\n",
      "Episode 1208\tAverage Score: 0.13\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.4031468856610473, Memory size: 34828 and Num Steps: 69\n",
      "Episode 1209\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.4027437387753862, Memory size: 34879 and Num Steps: 51\n",
      "Episode 1210\tAverage Score: 0.13\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.40234099503661086, Memory size: 35009 and Num Steps: 130\n",
      "Episode 1211\tAverage Score: 0.13\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.40193865404157425, Memory size: 35081 and Num Steps: 72\n",
      "Episode 1212\tAverage Score: 0.13\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.40153671538753266, Memory size: 35149 and Num Steps: 68\n",
      "Episode 1213\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.40113517867214515, Memory size: 35202 and Num Steps: 53\n",
      "Episode 1214\tAverage Score: 0.13\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.40073404349347297, Memory size: 35225 and Num Steps: 23\n",
      "Episode 1215\tAverage Score: 0.13\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.4003333094499795, Memory size: 35334 and Num Steps: 109\n",
      "Episode 1216\tAverage Score: 0.14\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.39993297614052953, Memory size: 35554 and Num Steps: 220\n",
      "Episode 1217\tAverage Score: 0.14\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.39953304316438903, Memory size: 35605 and Num Steps: 51\n",
      "Episode 1218\tAverage Score: 0.14\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3991335101212246, Memory size: 35673 and Num Steps: 68\n",
      "Episode 1219\tAverage Score: 0.14\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3987343766111034, Memory size: 35727 and Num Steps: 54\n",
      "Episode 1220\tAverage Score: 0.14\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.3983356422344923, Memory size: 35852 and Num Steps: 125\n",
      "Episode 1221\tAverage Score: 0.14\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3979373065922578, Memory size: 35875 and Num Steps: 23\n",
      "Episode 1222\tAverage Score: 0.14\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3975393692856655, Memory size: 35924 and Num Steps: 49\n",
      "Episode 1223\tAverage Score: 0.15\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.39714182991637986, Memory size: 36089 and Num Steps: 165\n",
      "Episode 1224\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3967446880864635, Memory size: 36151 and Num Steps: 62\n",
      "Episode 1225\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.39634794339837703, Memory size: 36187 and Num Steps: 36\n",
      "Episode 1226\tAverage Score: 0.14\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.39595159545497866, Memory size: 36269 and Num Steps: 82\n",
      "Episode 1227\tAverage Score: 0.14\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3955556438595237, Memory size: 36300 and Num Steps: 31\n",
      "Episode 1228\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3951600882156642, Memory size: 36356 and Num Steps: 56\n",
      "Episode 1229\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.39476492812744846, Memory size: 36387 and Num Steps: 31\n",
      "Episode 1230\tAverage Score: 0.15\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.39437016319932106, Memory size: 36505 and Num Steps: 118\n",
      "Episode 1231\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.39397579303612174, Memory size: 36585 and Num Steps: 80\n",
      "Episode 1232\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3935818172430856, Memory size: 36616 and Num Steps: 31\n",
      "Episode 1233\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3931882354258425, Memory size: 36668 and Num Steps: 52\n",
      "Episode 1234\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.39279504719041664, Memory size: 36758 and Num Steps: 90\n",
      "Episode 1235\tAverage Score: 0.14\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.39240225214322627, Memory size: 36772 and Num Steps: 14\n",
      "Episode 1236\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.392009849891083, Memory size: 36851 and Num Steps: 79\n",
      "Episode 1237\tAverage Score: 0.14\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.39161784004119193, Memory size: 36901 and Num Steps: 50\n",
      "Episode 1238\tAverage Score: 0.14\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.3912262222011507, Memory size: 37010 and Num Steps: 109\n",
      "Episode 1239\tAverage Score: 0.15\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.3908349959789496, Memory size: 37136 and Num Steps: 126\n",
      "Episode 1240\tAverage Score: 0.15\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.39044416098297063, Memory size: 37245 and Num Steps: 109\n",
      "Episode 1241\tAverage Score: 0.15\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.3900537168219877, Memory size: 37390 and Num Steps: 145\n",
      "Episode 1242\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3896636631051657, Memory size: 37462 and Num Steps: 72\n",
      "Episode 1243\tAverage Score: 0.15\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.38927399944206054, Memory size: 37493 and Num Steps: 31\n",
      "Episode 1244\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.38888472544261843, Memory size: 37573 and Num Steps: 80\n",
      "Episode 1245\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.38849584071717586, Memory size: 37623 and Num Steps: 50\n",
      "Episode 1246\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.38810734487645865, Memory size: 37710 and Num Steps: 87\n",
      "Episode 1247\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3877192375315822, Memory size: 37759 and Num Steps: 49\n",
      "Episode 1248\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3873315182940506, Memory size: 37810 and Num Steps: 51\n",
      "Episode 1249\tAverage Score: 0.15\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.3869441867757566, Memory size: 37824 and Num Steps: 14\n",
      "Episode 1250\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.38655724258898083, Memory size: 37876 and Num Steps: 52\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1251\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3861706853463918, Memory size: 37951 and Num Steps: 75\n",
      "Episode 1252\tAverage Score: 0.15\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.38578451466104546, Memory size: 38112 and Num Steps: 161\n",
      "Episode 1253\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.38539873014638437, Memory size: 38149 and Num Steps: 37\n",
      "Episode 1254\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.385013331416238, Memory size: 38220 and Num Steps: 71\n",
      "Episode 1255\tAverage Score: 0.15\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.3846283180848218, Memory size: 38328 and Num Steps: 108\n",
      "Episode 1256\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.38424368976673695, Memory size: 38378 and Num Steps: 50\n",
      "Episode 1257\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3838594460769702, Memory size: 38442 and Num Steps: 64\n",
      "Episode 1258\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.38347558663089326, Memory size: 38493 and Num Steps: 51\n",
      "Episode 1259\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.38309211104426233, Memory size: 38582 and Num Steps: 89\n",
      "Episode 1260\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.38270901893321807, Memory size: 38654 and Num Steps: 72\n",
      "Episode 1261\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.38232630991428485, Memory size: 38696 and Num Steps: 42\n",
      "Episode 1262\tAverage Score: 0.16\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.3819439836043706, Memory size: 38882 and Num Steps: 186\n",
      "Episode 1263\tAverage Score: 0.16\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.38156203962076624, Memory size: 38950 and Num Steps: 68\n",
      "Episode 1264\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3811804775811454, Memory size: 39018 and Num Steps: 68\n",
      "Episode 1265\tAverage Score: 0.17\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.3807992971035643, Memory size: 39243 and Num Steps: 225\n",
      "Episode 1266\tAverage Score: 0.17\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.38041849780646075, Memory size: 39352 and Num Steps: 109\n",
      "Episode 1267\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3800380793086543, Memory size: 39403 and Num Steps: 51\n",
      "Episode 1268\tAverage Score: 0.17\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.3796580412293456, Memory size: 39528 and Num Steps: 125\n",
      "Episode 1269\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3792783831881163, Memory size: 39565 and Num Steps: 37\n",
      "Episode 1270\tAverage Score: 0.16\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.37889910480492817, Memory size: 39579 and Num Steps: 14\n",
      "Episode 1271\tAverage Score: 0.17\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.37852020570012324, Memory size: 39792 and Num Steps: 213\n",
      "Episode 1272\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3781416854944231, Memory size: 39844 and Num Steps: 52\n",
      "Episode 1273\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3777635438089287, Memory size: 39915 and Num Steps: 71\n",
      "Episode 1274\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.37738578026511976, Memory size: 39967 and Num Steps: 52\n",
      "Episode 1275\tAverage Score: 0.17\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.3770083944848546, Memory size: 40042 and Num Steps: 75\n",
      "Episode 1276\tAverage Score: 0.17\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.3766313860903698, Memory size: 40151 and Num Steps: 109\n",
      "Episode 1277\tAverage Score: 0.17\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.3762547547042794, Memory size: 40165 and Num Steps: 14\n",
      "Episode 1278\tAverage Score: 0.17\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3758784999495751, Memory size: 40220 and Num Steps: 55\n",
      "Episode 1279\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.37550262144962554, Memory size: 40269 and Num Steps: 49\n",
      "Episode 1280\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3751271188281759, Memory size: 40320 and Num Steps: 51\n",
      "Episode 1281\tAverage Score: 0.17\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.37475199170934775, Memory size: 40392 and Num Steps: 72\n",
      "Episode 1282\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3743772397176384, Memory size: 40440 and Num Steps: 48\n",
      "Episode 1283\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.37400286247792075, Memory size: 40492 and Num Steps: 52\n",
      "Episode 1284\tAverage Score: 0.17\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.37362885961544284, Memory size: 40562 and Num Steps: 70\n",
      "Episode 1285\tAverage Score: 0.17\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.3732552307558274, Memory size: 40743 and Num Steps: 181\n",
      "Episode 1286\tAverage Score: 0.17\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3728819755250716, Memory size: 40833 and Num Steps: 90\n",
      "Episode 1287\tAverage Score: 0.17\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3725090935495465, Memory size: 40922 and Num Steps: 89\n",
      "Episode 1288\tAverage Score: 0.17\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.37213658445599695, Memory size: 41048 and Num Steps: 126\n",
      "Episode 1289\tAverage Score: 0.17\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.37176444787154095, Memory size: 41074 and Num Steps: 26\n",
      "Episode 1290\tAverage Score: 0.17\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.37139268342366943, Memory size: 41141 and Num Steps: 67\n",
      "Episode 1291\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.37102129074024576, Memory size: 41188 and Num Steps: 47\n",
      "Episode 1292\tAverage Score: 0.17\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.3706502694495055, Memory size: 41254 and Num Steps: 66\n",
      "Episode 1293\tAverage Score: 0.18\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.370279619180056, Memory size: 41363 and Num Steps: 109\n",
      "Episode 1294\tAverage Score: 0.18\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3699093395608759, Memory size: 41434 and Num Steps: 71\n",
      "Episode 1295\tAverage Score: 0.18\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3695394302213151, Memory size: 41522 and Num Steps: 88\n",
      "Episode 1296\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.36916989079109375, Memory size: 41553 and Num Steps: 31\n",
      "Episode 1297\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.36880072090030264, Memory size: 41604 and Num Steps: 51\n",
      "Episode 1298\tAverage Score: 0.18\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.36843192017940235, Memory size: 41694 and Num Steps: 90\n",
      "Episode 1299\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3680634882592229, Memory size: 41746 and Num Steps: 52\n",
      "Episode 1300\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.36769542477096373, Memory size: 41797 and Num Steps: 51\n",
      "Episode 1301\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.36732772934619273, Memory size: 41849 and Num Steps: 52\n",
      "Episode 1302\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.36696040161684657, Memory size: 41880 and Num Steps: 31\n",
      "Episode 1303\tAverage Score: 0.18\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.3665934412152297, Memory size: 41985 and Num Steps: 105\n",
      "Episode 1304\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3662268477740145, Memory size: 42037 and Num Steps: 52\n",
      "Episode 1305\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.36586062092624044, Memory size: 42089 and Num Steps: 52\n",
      "Episode 1306\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.36549476030531425, Memory size: 42138 and Num Steps: 49\n",
      "Episode 1307\tAverage Score: 0.18\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3651292655450089, Memory size: 42229 and Num Steps: 91\n",
      "Episode 1308\tAverage Score: 0.18\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3647641362794639, Memory size: 42297 and Num Steps: 68\n",
      "Episode 1309\tAverage Score: 0.18\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.36439937214318446, Memory size: 42364 and Num Steps: 67\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1310\tAverage Score: 0.18\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.36403497277104124, Memory size: 42381 and Num Steps: 17\n",
      "Episode 1311\tAverage Score: 0.18\tCurrent Score: 0.2900000046938658\n",
      "Noise Scaling: 0.3636709377982702, Memory size: 42487 and Num Steps: 106\n",
      "Episode 1312\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.36330726686047193, Memory size: 42538 and Num Steps: 51\n",
      "Episode 1313\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3629439595936115, Memory size: 42571 and Num Steps: 33\n",
      "Episode 1314\tAverage Score: 0.18\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.36258101563401784, Memory size: 42585 and Num Steps: 14\n",
      "Episode 1315\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3622184346183838, Memory size: 42637 and Num Steps: 52\n",
      "Episode 1316\tAverage Score: 0.17\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.36185621618376546, Memory size: 42651 and Num Steps: 14\n",
      "Episode 1317\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.36149435996758167, Memory size: 42704 and Num Steps: 53\n",
      "Episode 1318\tAverage Score: 0.17\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3611328656076141, Memory size: 42775 and Num Steps: 71\n",
      "Episode 1319\tAverage Score: 0.17\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.36077173274200647, Memory size: 42842 and Num Steps: 67\n",
      "Episode 1320\tAverage Score: 0.17\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.3604109610092645, Memory size: 42923 and Num Steps: 81\n",
      "Episode 1321\tAverage Score: 0.17\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3600505500482552, Memory size: 42994 and Num Steps: 71\n",
      "Episode 1322\tAverage Score: 0.18\tCurrent Score: 0.5900000091642141\n",
      "Noise Scaling: 0.359690499498207, Memory size: 43216 and Num Steps: 222\n",
      "Episode 1323\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3593308089987088, Memory size: 43274 and Num Steps: 58\n",
      "Episode 1324\tAverage Score: 0.17\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.35897147818971004, Memory size: 43364 and Num Steps: 90\n",
      "Episode 1325\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.35861250671152034, Memory size: 43415 and Num Steps: 51\n",
      "Episode 1326\tAverage Score: 0.18\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.3582538942048088, Memory size: 43544 and Num Steps: 129\n",
      "Episode 1327\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.357895640310604, Memory size: 43597 and Num Steps: 53\n",
      "Episode 1328\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3575377446702934, Memory size: 43649 and Num Steps: 52\n",
      "Episode 1329\tAverage Score: 0.18\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.35718020692562313, Memory size: 43714 and Num Steps: 65\n",
      "Episode 1330\tAverage Score: 0.18\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.3568230267186975, Memory size: 43842 and Num Steps: 128\n",
      "Episode 1331\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3564662036919788, Memory size: 43870 and Num Steps: 28\n",
      "Episode 1332\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.35610973748828684, Memory size: 43921 and Num Steps: 51\n",
      "Episode 1333\tAverage Score: 0.18\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.35575362775079855, Memory size: 43955 and Num Steps: 34\n",
      "Episode 1334\tAverage Score: 0.18\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.35539787412304774, Memory size: 44022 and Num Steps: 67\n",
      "Episode 1335\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.35504247624892465, Memory size: 44072 and Num Steps: 50\n",
      "Episode 1336\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.35468743377267575, Memory size: 44138 and Num Steps: 66\n",
      "Episode 1337\tAverage Score: 0.18\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3543327463389031, Memory size: 44208 and Num Steps: 70\n",
      "Episode 1338\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.35397841359256416, Memory size: 44259 and Num Steps: 51\n",
      "Episode 1339\tAverage Score: 0.17\tCurrent Score: 0.3900000061839819\n",
      "Noise Scaling: 0.35362443517897163, Memory size: 44329 and Num Steps: 70\n",
      "Episode 1340\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.35327081074379263, Memory size: 44381 and Num Steps: 52\n",
      "Episode 1341\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.35291753993304886, Memory size: 44432 and Num Steps: 51\n",
      "Episode 1342\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3525646223931158, Memory size: 44499 and Num Steps: 67\n",
      "Episode 1343\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3522120577707227, Memory size: 44548 and Num Steps: 49\n",
      "Episode 1344\tAverage Score: 0.17\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.351859845712952, Memory size: 44649 and Num Steps: 101\n",
      "Episode 1345\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.351507985867239, Memory size: 44680 and Num Steps: 31\n",
      "Episode 1346\tAverage Score: 0.17\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.3511564778813718, Memory size: 44847 and Num Steps: 167\n",
      "Episode 1347\tAverage Score: 0.17\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.3508053214034904, Memory size: 44860 and Num Steps: 13\n",
      "Episode 1348\tAverage Score: 0.17\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3504545160820869, Memory size: 44950 and Num Steps: 90\n",
      "Episode 1349\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3501040615660048, Memory size: 45004 and Num Steps: 54\n",
      "Episode 1350\tAverage Score: 0.17\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.34975395750443883, Memory size: 45099 and Num Steps: 95\n",
      "Episode 1351\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.34940420354693436, Memory size: 45124 and Num Steps: 25\n",
      "Episode 1352\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.34905479934338746, Memory size: 45179 and Num Steps: 55\n",
      "Episode 1353\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.34870574454404407, Memory size: 45231 and Num Steps: 52\n",
      "Episode 1354\tAverage Score: 0.17\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.34835703879950003, Memory size: 45394 and Num Steps: 163\n",
      "Episode 1355\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3480086817607005, Memory size: 45425 and Num Steps: 31\n",
      "Episode 1356\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.34766067307893983, Memory size: 45456 and Num Steps: 31\n",
      "Episode 1357\tAverage Score: 0.17\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.3473130124058609, Memory size: 45583 and Num Steps: 127\n",
      "Episode 1358\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.346965699393455, Memory size: 45635 and Num Steps: 52\n",
      "Episode 1359\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.34661873369406154, Memory size: 45684 and Num Steps: 49\n",
      "Episode 1360\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.34627211496036747, Memory size: 45719 and Num Steps: 35\n",
      "Episode 1361\tAverage Score: 0.17\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3459258428454071, Memory size: 45789 and Num Steps: 70\n",
      "Episode 1362\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3455799170025617, Memory size: 45841 and Num Steps: 52\n",
      "Episode 1363\tAverage Score: 0.17\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.3452343370855592, Memory size: 45949 and Num Steps: 108\n",
      "Episode 1364\tAverage Score: 0.17\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.34488910274847356, Memory size: 46003 and Num Steps: 54\n",
      "Episode 1365\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.34454421364572513, Memory size: 46033 and Num Steps: 30\n",
      "Episode 1366\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3441996694320794, Memory size: 46085 and Num Steps: 52\n",
      "Episode 1367\tAverage Score: 0.16\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.3438554697626473, Memory size: 46215 and Num Steps: 130\n",
      "Episode 1368\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3435116142928847, Memory size: 46247 and Num Steps: 32\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1369\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.34316810267859177, Memory size: 46315 and Num Steps: 68\n",
      "Episode 1370\tAverage Score: 0.16\tCurrent Score: 0.3900000061839819\n",
      "Noise Scaling: 0.3428249345759132, Memory size: 46459 and Num Steps: 144\n",
      "Episode 1371\tAverage Score: 0.16\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.3424821096413373, Memory size: 46665 and Num Steps: 206\n",
      "Episode 1372\tAverage Score: 0.16\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.3421396275316959, Memory size: 46698 and Num Steps: 33\n",
      "Episode 1373\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.34179748790416425, Memory size: 46749 and Num Steps: 51\n",
      "Episode 1374\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.34145569041626006, Memory size: 46839 and Num Steps: 90\n",
      "Episode 1375\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3411142347258438, Memory size: 46907 and Num Steps: 68\n",
      "Episode 1376\tAverage Score: 0.16\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.34077312049111796, Memory size: 46974 and Num Steps: 67\n",
      "Episode 1377\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.34043234737062683, Memory size: 47027 and Num Steps: 53\n",
      "Episode 1378\tAverage Score: 0.16\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.3400919150232562, Memory size: 47041 and Num Steps: 14\n",
      "Episode 1379\tAverage Score: 0.16\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.339751823108233, Memory size: 47074 and Num Steps: 33\n",
      "Episode 1380\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.33941207128512474, Memory size: 47117 and Num Steps: 43\n",
      "Episode 1381\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3390726592138396, Memory size: 47177 and Num Steps: 60\n",
      "Episode 1382\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.33873358655462577, Memory size: 47231 and Num Steps: 54\n",
      "Episode 1383\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.33839485296807115, Memory size: 47283 and Num Steps: 52\n",
      "Episode 1384\tAverage Score: 0.17\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.33805645811510304, Memory size: 47574 and Num Steps: 291\n",
      "Episode 1385\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.33771840165698797, Memory size: 47625 and Num Steps: 51\n",
      "Episode 1386\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.337380683255331, Memory size: 47658 and Num Steps: 33\n",
      "Episode 1387\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.33704330257207565, Memory size: 47710 and Num Steps: 52\n",
      "Episode 1388\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3367062592695036, Memory size: 47797 and Num Steps: 87\n",
      "Episode 1389\tAverage Score: 0.16\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.3363695530102341, Memory size: 47811 and Num Steps: 14\n",
      "Episode 1390\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3360331834572238, Memory size: 47898 and Num Steps: 87\n",
      "Episode 1391\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3356971502737666, Memory size: 47928 and Num Steps: 30\n",
      "Episode 1392\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.33536145312349286, Memory size: 47981 and Num Steps: 53\n",
      "Episode 1393\tAverage Score: 0.16\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.33502609167036934, Memory size: 48147 and Num Steps: 166\n",
      "Episode 1394\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.334691065578699, Memory size: 48236 and Num Steps: 89\n",
      "Episode 1395\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3343563745131203, Memory size: 48287 and Num Steps: 51\n",
      "Episode 1396\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.33402201813860716, Memory size: 48339 and Num Steps: 52\n",
      "Episode 1397\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.33368799612046857, Memory size: 48391 and Num Steps: 52\n",
      "Episode 1398\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3333543081243481, Memory size: 48443 and Num Steps: 52\n",
      "Episode 1399\tAverage Score: 0.16\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.3330209538162237, Memory size: 48571 and Num Steps: 128\n",
      "Episode 1400\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3326879328624075, Memory size: 48601 and Num Steps: 30\n",
      "Episode 1401\tAverage Score: 0.16\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.3323552449295451, Memory size: 48615 and Num Steps: 14\n",
      "Episode 1402\tAverage Score: 0.16\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.3320228896846156, Memory size: 48629 and Num Steps: 14\n",
      "Episode 1403\tAverage Score: 0.16\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.3316908667949309, Memory size: 48659 and Num Steps: 30\n",
      "Episode 1404\tAverage Score: 0.16\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.331359175928136, Memory size: 48726 and Num Steps: 67\n",
      "Episode 1405\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3310278167522079, Memory size: 48757 and Num Steps: 31\n",
      "Episode 1406\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3306967889354557, Memory size: 48808 and Num Steps: 51\n",
      "Episode 1407\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3303660921465202, Memory size: 48840 and Num Steps: 32\n",
      "Episode 1408\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3300357260543737, Memory size: 48911 and Num Steps: 71\n",
      "Episode 1409\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3297056903283193, Memory size: 48966 and Num Steps: 55\n",
      "Episode 1410\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.329375984637991, Memory size: 49058 and Num Steps: 92\n",
      "Episode 1411\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.329046608653353, Memory size: 49088 and Num Steps: 30\n",
      "Episode 1412\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.32871756204469965, Memory size: 49121 and Num Steps: 33\n",
      "Episode 1413\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.32838884448265493, Memory size: 49189 and Num Steps: 68\n",
      "Episode 1414\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3280604556381723, Memory size: 49219 and Num Steps: 30\n",
      "Episode 1415\tAverage Score: 0.16\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.3277323951825341, Memory size: 49268 and Num Steps: 49\n",
      "Episode 1416\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3274046627873516, Memory size: 49336 and Num Steps: 68\n",
      "Episode 1417\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.32707725812456423, Memory size: 49380 and Num Steps: 44\n",
      "Episode 1418\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3267501808664397, Memory size: 49443 and Num Steps: 63\n",
      "Episode 1419\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3264234306855732, Memory size: 49495 and Num Steps: 52\n",
      "Episode 1420\tAverage Score: 0.16\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.32609700725488766, Memory size: 49525 and Num Steps: 30\n",
      "Episode 1421\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3257709102476328, Memory size: 49626 and Num Steps: 101\n",
      "Episode 1422\tAverage Score: 0.15\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.3254451393373851, Memory size: 49640 and Num Steps: 14\n",
      "Episode 1423\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.32511969419804776, Memory size: 49692 and Num Steps: 52\n",
      "Episode 1424\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3247945745038497, Memory size: 49744 and Num Steps: 52\n",
      "Episode 1425\tAverage Score: 0.15\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.32446977992934584, Memory size: 49872 and Num Steps: 128\n",
      "Episode 1426\tAverage Score: 0.15\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.3241453101494165, Memory size: 50037 and Num Steps: 165\n",
      "Episode 1427\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3238211648392671, Memory size: 50090 and Num Steps: 53\n",
      "Episode 1428\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.32349734367442784, Memory size: 50141 and Num Steps: 51\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1429\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3231738463307534, Memory size: 50192 and Num Steps: 51\n",
      "Episode 1430\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3228506724844226, Memory size: 50286 and Num Steps: 94\n",
      "Episode 1431\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.32252782181193823, Memory size: 50316 and Num Steps: 30\n",
      "Episode 1432\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.32220529399012626, Memory size: 50372 and Num Steps: 56\n",
      "Episode 1433\tAverage Score: 0.15\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.32188308869613613, Memory size: 50440 and Num Steps: 68\n",
      "Episode 1434\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.32156120560744, Memory size: 50470 and Num Steps: 30\n",
      "Episode 1435\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.32123964440183256, Memory size: 50492 and Num Steps: 22\n",
      "Episode 1436\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3209184047574307, Memory size: 50562 and Num Steps: 70\n",
      "Episode 1437\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3205974863526733, Memory size: 50630 and Num Steps: 68\n",
      "Episode 1438\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.32027688886632066, Memory size: 50681 and Num Steps: 51\n",
      "Episode 1439\tAverage Score: 0.15\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.31995661197745434, Memory size: 50791 and Num Steps: 110\n",
      "Episode 1440\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.31963665536547686, Memory size: 50843 and Num Steps: 52\n",
      "Episode 1441\tAverage Score: 0.15\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.31931701871011137, Memory size: 50910 and Num Steps: 67\n",
      "Episode 1442\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3189977016914013, Memory size: 50999 and Num Steps: 89\n",
      "Episode 1443\tAverage Score: 0.15\tCurrent Score: 0.2900000046938658\n",
      "Noise Scaling: 0.3186787039897099, Memory size: 51106 and Num Steps: 107\n",
      "Episode 1444\tAverage Score: 0.16\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.31836002528572016, Memory size: 51236 and Num Steps: 130\n",
      "Episode 1445\tAverage Score: 0.16\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.3180416652604344, Memory size: 51286 and Num Steps: 50\n",
      "Episode 1446\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.31772362359517403, Memory size: 51403 and Num Steps: 117\n",
      "Episode 1447\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3174058999715788, Memory size: 51454 and Num Steps: 51\n",
      "Episode 1448\tAverage Score: 0.15\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.3170884940716073, Memory size: 51474 and Num Steps: 20\n",
      "Episode 1449\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.31677140557753564, Memory size: 51544 and Num Steps: 70\n",
      "Episode 1450\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3164546341719581, Memory size: 51633 and Num Steps: 89\n",
      "Episode 1451\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.31613817953778617, Memory size: 51724 and Num Steps: 91\n",
      "Episode 1452\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3158220413582484, Memory size: 51815 and Num Steps: 91\n",
      "Episode 1453\tAverage Score: 0.15\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.3155062193168901, Memory size: 51829 and Num Steps: 14\n",
      "Episode 1454\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.31519071309757324, Memory size: 51881 and Num Steps: 52\n",
      "Episode 1455\tAverage Score: 0.15\tCurrent Score: 0.2900000046938658\n",
      "Noise Scaling: 0.31487552238447564, Memory size: 51986 and Num Steps: 105\n",
      "Episode 1456\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3145606468620912, Memory size: 52037 and Num Steps: 51\n",
      "Episode 1457\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3142460862152291, Memory size: 52089 and Num Steps: 52\n",
      "Episode 1458\tAverage Score: 0.15\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3139318401290139, Memory size: 52178 and Num Steps: 89\n",
      "Episode 1459\tAverage Score: 0.15\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.31361790828888486, Memory size: 52306 and Num Steps: 128\n",
      "Episode 1460\tAverage Score: 0.16\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.313304290380596, Memory size: 52452 and Num Steps: 146\n",
      "Episode 1461\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.31299098609021536, Memory size: 52519 and Num Steps: 67\n",
      "Episode 1462\tAverage Score: 0.16\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.31267799510412514, Memory size: 52532 and Num Steps: 13\n",
      "Episode 1463\tAverage Score: 0.15\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.31236531710902105, Memory size: 52584 and Num Steps: 52\n",
      "Episode 1464\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.312052951791912, Memory size: 52655 and Num Steps: 71\n",
      "Episode 1465\tAverage Score: 0.16\tCurrent Score: 0.2900000046938658\n",
      "Noise Scaling: 0.3117408988401201, Memory size: 52759 and Num Steps: 104\n",
      "Episode 1466\tAverage Score: 0.16\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.31142915794127995, Memory size: 52828 and Num Steps: 69\n",
      "Episode 1467\tAverage Score: 0.15\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.3111177287833387, Memory size: 52842 and Num Steps: 14\n",
      "Episode 1468\tAverage Score: 0.16\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.31080661105455537, Memory size: 52950 and Num Steps: 108\n",
      "Episode 1469\tAverage Score: 0.17\tCurrent Score: 1.600000023841858\n",
      "Noise Scaling: 0.3104958044435008, Memory size: 53574 and Num Steps: 624\n",
      "Episode 1470\tAverage Score: 0.17\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.3101853086390573, Memory size: 53793 and Num Steps: 219\n",
      "Episode 1471\tAverage Score: 0.17\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.3098751233304182, Memory size: 53916 and Num Steps: 123\n",
      "Episode 1472\tAverage Score: 0.18\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.3095652482070878, Memory size: 54119 and Num Steps: 203\n",
      "Episode 1473\tAverage Score: 0.18\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.30925568295888073, Memory size: 54245 and Num Steps: 126\n",
      "Episode 1474\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.30894642727592186, Memory size: 54296 and Num Steps: 51\n",
      "Episode 1475\tAverage Score: 0.18\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.30863748084864595, Memory size: 54448 and Num Steps: 152\n",
      "Episode 1476\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.30832884336779726, Memory size: 54482 and Num Steps: 34\n",
      "Episode 1477\tAverage Score: 0.18\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.30802051452442947, Memory size: 54587 and Num Steps: 105\n",
      "Episode 1478\tAverage Score: 0.18\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.307712494009905, Memory size: 54708 and Num Steps: 121\n",
      "Episode 1479\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3074047815158951, Memory size: 54761 and Num Steps: 53\n",
      "Episode 1480\tAverage Score: 0.18\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.30709737673437926, Memory size: 54830 and Num Steps: 69\n",
      "Episode 1481\tAverage Score: 0.18\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.30679027935764486, Memory size: 54911 and Num Steps: 81\n",
      "Episode 1482\tAverage Score: 0.18\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3064834890782872, Memory size: 54962 and Num Steps: 51\n",
      "Episode 1483\tAverage Score: 0.19\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.30617700558920896, Memory size: 55033 and Num Steps: 71\n",
      "Episode 1484\tAverage Score: 0.18\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.30587082858361975, Memory size: 55237 and Num Steps: 204\n",
      "Episode 1485\tAverage Score: 0.19\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.3055649577550361, Memory size: 55378 and Num Steps: 141\n",
      "Episode 1486\tAverage Score: 0.19\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.3052593927972811, Memory size: 55563 and Num Steps: 185\n",
      "Episode 1487\tAverage Score: 0.19\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3049541334044838, Memory size: 55615 and Num Steps: 52\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1488\tAverage Score: 0.19\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3046491792710793, Memory size: 55666 and Num Steps: 51\n",
      "Episode 1489\tAverage Score: 0.19\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.30434453009180823, Memory size: 55745 and Num Steps: 79\n",
      "Episode 1490\tAverage Score: 0.19\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3040401855617164, Memory size: 55776 and Num Steps: 31\n",
      "Episode 1491\tAverage Score: 0.19\tCurrent Score: 0.2900000046938658\n",
      "Noise Scaling: 0.3037361453761547, Memory size: 55881 and Num Steps: 105\n",
      "Episode 1492\tAverage Score: 0.19\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.3034324092307785, Memory size: 56006 and Num Steps: 125\n",
      "Episode 1493\tAverage Score: 0.19\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.30312897682154777, Memory size: 56077 and Num Steps: 71\n",
      "Episode 1494\tAverage Score: 0.19\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.3028258478447262, Memory size: 56187 and Num Steps: 110\n",
      "Episode 1495\tAverage Score: 0.19\tCurrent Score: 0.3900000061839819\n",
      "Noise Scaling: 0.30252302199688147, Memory size: 56333 and Num Steps: 146\n",
      "Episode 1496\tAverage Score: 0.19\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3022204989748846, Memory size: 56385 and Num Steps: 52\n",
      "Episode 1497\tAverage Score: 0.19\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.30191827847590974, Memory size: 56416 and Num Steps: 31\n",
      "Episode 1498\tAverage Score: 0.19\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3016163601974338, Memory size: 56435 and Num Steps: 19\n",
      "Episode 1499\tAverage Score: 0.19\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3013147438372364, Memory size: 56488 and Num Steps: 53\n",
      "Episode 1500\tAverage Score: 0.20\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.3010134290933991, Memory size: 56652 and Num Steps: 164\n",
      "Saved Model: Episode 1500\tAverage Score: 0.20\n",
      "Episode 1501\tAverage Score: 0.20\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.30071241566430573, Memory size: 56723 and Num Steps: 71\n",
      "Episode 1502\tAverage Score: 0.20\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.3004117032486414, Memory size: 56818 and Num Steps: 95\n",
      "Episode 1503\tAverage Score: 0.20\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.3001112915453928, Memory size: 56849 and Num Steps: 31\n",
      "Episode 1504\tAverage Score: 0.20\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2998111802538474, Memory size: 56938 and Num Steps: 89\n",
      "Episode 1505\tAverage Score: 0.20\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.29951136907359355, Memory size: 57027 and Num Steps: 89\n",
      "Episode 1506\tAverage Score: 0.20\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.29921185770451997, Memory size: 57193 and Num Steps: 166\n",
      "Episode 1507\tAverage Score: 0.21\tCurrent Score: 0.800000011920929\n",
      "Noise Scaling: 0.29891264584681543, Memory size: 57491 and Num Steps: 298\n",
      "Episode 1508\tAverage Score: 0.21\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.2986137332009686, Memory size: 57609 and Num Steps: 118\n",
      "Episode 1509\tAverage Score: 0.21\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2983151194677677, Memory size: 57661 and Num Steps: 52\n",
      "Episode 1510\tAverage Score: 0.21\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.2980168043482999, Memory size: 57788 and Num Steps: 127\n",
      "Episode 1511\tAverage Score: 0.21\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2977187875439516, Memory size: 57891 and Num Steps: 103\n",
      "Episode 1512\tAverage Score: 0.21\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2974210687564076, Memory size: 57981 and Num Steps: 90\n",
      "Episode 1513\tAverage Score: 0.21\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2971236476876512, Memory size: 58065 and Num Steps: 84\n",
      "Episode 1514\tAverage Score: 0.21\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.29682652403996357, Memory size: 58079 and Num Steps: 14\n",
      "Episode 1515\tAverage Score: 0.21\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2965296975159236, Memory size: 58117 and Num Steps: 38\n",
      "Episode 1516\tAverage Score: 0.21\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2962331678184077, Memory size: 58166 and Num Steps: 49\n",
      "Episode 1517\tAverage Score: 0.22\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.2959369346505893, Memory size: 58383 and Num Steps: 217\n",
      "Episode 1518\tAverage Score: 0.22\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2956409977159387, Memory size: 58413 and Num Steps: 30\n",
      "Episode 1519\tAverage Score: 0.22\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.29534535671822276, Memory size: 58467 and Num Steps: 54\n",
      "Episode 1520\tAverage Score: 0.22\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.29505001136150455, Memory size: 58533 and Num Steps: 66\n",
      "Episode 1521\tAverage Score: 0.22\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.294754961350143, Memory size: 58585 and Num Steps: 52\n",
      "Episode 1522\tAverage Score: 0.22\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.2944602063887929, Memory size: 58789 and Num Steps: 204\n",
      "Episode 1523\tAverage Score: 0.22\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2941657461824041, Memory size: 58842 and Num Steps: 53\n",
      "Episode 1524\tAverage Score: 0.22\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2938715804362217, Memory size: 58949 and Num Steps: 107\n",
      "Episode 1525\tAverage Score: 0.22\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.2935777088557855, Memory size: 59080 and Num Steps: 131\n",
      "Episode 1526\tAverage Score: 0.22\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.2932841311469297, Memory size: 59114 and Num Steps: 34\n",
      "Episode 1527\tAverage Score: 0.22\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.29299084701578276, Memory size: 59164 and Num Steps: 50\n",
      "Episode 1528\tAverage Score: 0.22\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.292697856168767, Memory size: 59205 and Num Steps: 41\n",
      "Episode 1529\tAverage Score: 0.22\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.2924051583125982, Memory size: 59330 and Num Steps: 125\n",
      "Episode 1530\tAverage Score: 0.22\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.2921127531542856, Memory size: 59439 and Num Steps: 109\n",
      "Episode 1531\tAverage Score: 0.22\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.2918206404011313, Memory size: 59453 and Num Steps: 14\n",
      "Episode 1532\tAverage Score: 0.22\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2915288197607302, Memory size: 59505 and Num Steps: 52\n",
      "Episode 1533\tAverage Score: 0.22\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.29123729094096945, Memory size: 59518 and Num Steps: 13\n",
      "Episode 1534\tAverage Score: 0.22\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.29094605365002846, Memory size: 59706 and Num Steps: 188\n",
      "Episode 1535\tAverage Score: 0.22\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.29065510759637847, Memory size: 59740 and Num Steps: 34\n",
      "Episode 1536\tAverage Score: 0.23\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.29036445248878207, Memory size: 60015 and Num Steps: 275\n",
      "Episode 1537\tAverage Score: 0.23\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2900740880362933, Memory size: 60067 and Num Steps: 52\n",
      "Episode 1538\tAverage Score: 0.23\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.289784013948257, Memory size: 60156 and Num Steps: 89\n",
      "Episode 1539\tAverage Score: 0.23\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.28949422993430873, Memory size: 60323 and Num Steps: 167\n",
      "Episode 1540\tAverage Score: 0.23\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.28920473570437444, Memory size: 60394 and Num Steps: 71\n",
      "Episode 1541\tAverage Score: 0.23\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.28891553096867006, Memory size: 60464 and Num Steps: 70\n",
      "Episode 1542\tAverage Score: 0.23\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2886266154377014, Memory size: 60517 and Num Steps: 53\n",
      "Episode 1543\tAverage Score: 0.23\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2883379888222637, Memory size: 60569 and Num Steps: 52\n",
      "Episode 1544\tAverage Score: 0.23\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.2880496508334414, Memory size: 60844 and Num Steps: 275\n",
      "Episode 1545\tAverage Score: 0.23\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.28776160118260796, Memory size: 60914 and Num Steps: 70\n",
      "Episode 1546\tAverage Score: 0.23\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.28747383958142536, Memory size: 60966 and Num Steps: 52\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1547\tAverage Score: 0.24\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.287186365741844, Memory size: 61190 and Num Steps: 224\n",
      "Episode 1548\tAverage Score: 0.24\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2868991793761021, Memory size: 61280 and Num Steps: 90\n",
      "Episode 1549\tAverage Score: 0.24\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.286612280196726, Memory size: 61441 and Num Steps: 161\n",
      "Episode 1550\tAverage Score: 0.24\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.2863256679165293, Memory size: 61589 and Num Steps: 148\n",
      "Episode 1551\tAverage Score: 0.24\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.28603934224861277, Memory size: 61642 and Num Steps: 53\n",
      "Episode 1552\tAverage Score: 0.24\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.28575330290636414, Memory size: 61829 and Num Steps: 187\n",
      "Episode 1553\tAverage Score: 0.25\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.28546754960345777, Memory size: 62112 and Num Steps: 283\n",
      "Episode 1554\tAverage Score: 0.25\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2851820820538543, Memory size: 62183 and Num Steps: 71\n",
      "Episode 1555\tAverage Score: 0.25\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.28489689997180045, Memory size: 62252 and Num Steps: 69\n",
      "Episode 1556\tAverage Score: 0.25\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.28461200307182866, Memory size: 62308 and Num Steps: 56\n",
      "Episode 1557\tAverage Score: 0.26\tCurrent Score: 0.5900000091642141\n",
      "Noise Scaling: 0.2843273910687568, Memory size: 62527 and Num Steps: 219\n",
      "Episode 1558\tAverage Score: 0.26\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.28404306367768806, Memory size: 62689 and Num Steps: 162\n",
      "Episode 1559\tAverage Score: 0.26\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.28375902061401037, Memory size: 62813 and Num Steps: 124\n",
      "Episode 1560\tAverage Score: 0.26\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.28347526159339637, Memory size: 62902 and Num Steps: 89\n",
      "Episode 1561\tAverage Score: 0.26\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.283191786331803, Memory size: 63133 and Num Steps: 231\n",
      "Episode 1562\tAverage Score: 0.26\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.2829085945454712, Memory size: 63276 and Num Steps: 143\n",
      "Episode 1563\tAverage Score: 0.26\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2826256859509257, Memory size: 63360 and Num Steps: 84\n",
      "Episode 1564\tAverage Score: 0.26\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2823430602649748, Memory size: 63411 and Num Steps: 51\n",
      "Episode 1565\tAverage Score: 0.26\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2820607172047098, Memory size: 63455 and Num Steps: 44\n",
      "Episode 1566\tAverage Score: 0.26\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.2817786564875051, Memory size: 63593 and Num Steps: 138\n",
      "Episode 1567\tAverage Score: 0.26\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.28149687783101757, Memory size: 63660 and Num Steps: 67\n",
      "Episode 1568\tAverage Score: 0.26\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.28121538095318654, Memory size: 63731 and Num Steps: 71\n",
      "Episode 1569\tAverage Score: 0.25\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.2809341655722334, Memory size: 63746 and Num Steps: 15\n",
      "Episode 1570\tAverage Score: 0.24\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.28065323140666115, Memory size: 63815 and Num Steps: 69\n",
      "Episode 1571\tAverage Score: 0.24\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.28037257817525446, Memory size: 63863 and Num Steps: 48\n",
      "Episode 1572\tAverage Score: 0.24\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.2800922055970792, Memory size: 63877 and Num Steps: 14\n",
      "Episode 1573\tAverage Score: 0.24\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.27981211339148215, Memory size: 64133 and Num Steps: 256\n",
      "Episode 1574\tAverage Score: 0.24\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.27953230127809064, Memory size: 64184 and Num Steps: 51\n",
      "Episode 1575\tAverage Score: 0.24\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.27925276897681256, Memory size: 64238 and Num Steps: 54\n",
      "Episode 1576\tAverage Score: 0.24\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2789735162078358, Memory size: 64272 and Num Steps: 34\n",
      "Episode 1577\tAverage Score: 0.23\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2786945426916279, Memory size: 64324 and Num Steps: 52\n",
      "Episode 1578\tAverage Score: 0.23\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2784158481489363, Memory size: 64413 and Num Steps: 89\n",
      "Episode 1579\tAverage Score: 0.24\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.27813743230078736, Memory size: 64621 and Num Steps: 208\n",
      "Episode 1580\tAverage Score: 0.24\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.2778592948684866, Memory size: 64691 and Num Steps: 70\n",
      "Episode 1581\tAverage Score: 0.24\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2775814355736181, Memory size: 64777 and Num Steps: 86\n",
      "Episode 1582\tAverage Score: 0.24\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2773038541380445, Memory size: 64809 and Num Steps: 32\n",
      "Episode 1583\tAverage Score: 0.24\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.27702655028390644, Memory size: 64914 and Num Steps: 105\n",
      "Episode 1584\tAverage Score: 0.24\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.2767495237336225, Memory size: 65177 and Num Steps: 263\n",
      "Episode 1585\tAverage Score: 0.24\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.2764727742098889, Memory size: 65424 and Num Steps: 247\n",
      "Episode 1586\tAverage Score: 0.24\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.276196301435679, Memory size: 65490 and Num Steps: 66\n",
      "Episode 1587\tAverage Score: 0.24\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2759201051342433, Memory size: 65558 and Num Steps: 68\n",
      "Episode 1588\tAverage Score: 0.24\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2756441850291091, Memory size: 65609 and Num Steps: 51\n",
      "Episode 1589\tAverage Score: 0.24\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.27536854084407997, Memory size: 65674 and Num Steps: 65\n",
      "Episode 1590\tAverage Score: 0.24\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2750931723032359, Memory size: 65727 and Num Steps: 53\n",
      "Episode 1591\tAverage Score: 0.24\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.27481807913093265, Memory size: 65778 and Num Steps: 51\n",
      "Episode 1592\tAverage Score: 0.24\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.2745432610518017, Memory size: 65883 and Num Steps: 105\n",
      "Episode 1593\tAverage Score: 0.24\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2742687177907499, Memory size: 65973 and Num Steps: 90\n",
      "Episode 1594\tAverage Score: 0.23\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.27399444907295917, Memory size: 65998 and Num Steps: 25\n",
      "Episode 1595\tAverage Score: 0.23\tCurrent Score: 0.1900000050663948\n",
      "Noise Scaling: 0.2737204546238862, Memory size: 66047 and Num Steps: 49\n",
      "Episode 1596\tAverage Score: 0.24\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.27344673416926235, Memory size: 66310 and Num Steps: 263\n",
      "Episode 1597\tAverage Score: 0.24\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.27317328743509306, Memory size: 66483 and Num Steps: 173\n",
      "Episode 1598\tAverage Score: 0.24\tCurrent Score: 0.2900000046938658\n",
      "Noise Scaling: 0.27290011414765797, Memory size: 66586 and Num Steps: 103\n",
      "Episode 1599\tAverage Score: 0.24\tCurrent Score: 0.2900000046938658\n",
      "Noise Scaling: 0.2726272140335103, Memory size: 66694 and Num Steps: 108\n",
      "Episode 1600\tAverage Score: 0.24\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.2723545868194768, Memory size: 66866 and Num Steps: 172\n",
      "Episode 1601\tAverage Score: 0.24\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.2720822322326573, Memory size: 66880 and Num Steps: 14\n",
      "Episode 1602\tAverage Score: 0.24\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.2718101500004247, Memory size: 66943 and Num Steps: 63\n",
      "Episode 1603\tAverage Score: 0.24\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.27153833985042425, Memory size: 66968 and Num Steps: 25\n",
      "Episode 1604\tAverage Score: 0.24\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2712668015105738, Memory size: 67020 and Num Steps: 52\n",
      "Episode 1605\tAverage Score: 0.24\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.27099553470906323, Memory size: 67045 and Num Steps: 25\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1606\tAverage Score: 0.24\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.2707245391743542, Memory size: 67059 and Num Steps: 14\n",
      "Episode 1607\tAverage Score: 0.23\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.2704538146351798, Memory size: 67190 and Num Steps: 131\n",
      "Episode 1608\tAverage Score: 0.23\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.27018336082054467, Memory size: 67257 and Num Steps: 67\n",
      "Episode 1609\tAverage Score: 0.23\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.2699131774597241, Memory size: 67458 and Num Steps: 201\n",
      "Episode 1610\tAverage Score: 0.23\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.26964326428226437, Memory size: 67639 and Num Steps: 181\n",
      "Episode 1611\tAverage Score: 0.24\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.2693736210179821, Memory size: 67785 and Num Steps: 146\n",
      "Episode 1612\tAverage Score: 0.23\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.26910424739696415, Memory size: 67838 and Num Steps: 53\n",
      "Episode 1613\tAverage Score: 0.23\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2688351431495672, Memory size: 67890 and Num Steps: 52\n",
      "Episode 1614\tAverage Score: 0.23\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2685663080064176, Memory size: 67942 and Num Steps: 52\n",
      "Episode 1615\tAverage Score: 0.24\tCurrent Score: 0.1900000050663948\n",
      "Noise Scaling: 0.2682977416984112, Memory size: 68010 and Num Steps: 68\n",
      "Episode 1616\tAverage Score: 0.24\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.26802944395671274, Memory size: 68053 and Num Steps: 43\n",
      "Episode 1617\tAverage Score: 0.23\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.26776141451275604, Memory size: 68105 and Num Steps: 52\n",
      "Episode 1618\tAverage Score: 0.23\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.2674936530982433, Memory size: 68119 and Num Steps: 14\n",
      "Episode 1619\tAverage Score: 0.23\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.26722615944514505, Memory size: 68229 and Num Steps: 110\n",
      "Episode 1620\tAverage Score: 0.23\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.2669589332856999, Memory size: 68373 and Num Steps: 144\n",
      "Episode 1621\tAverage Score: 0.24\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.2666919743524142, Memory size: 68577 and Num Steps: 204\n",
      "Episode 1622\tAverage Score: 0.24\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2664252823780618, Memory size: 68666 and Num Steps: 89\n",
      "Episode 1623\tAverage Score: 0.24\tCurrent Score: 0.3900000061839819\n",
      "Noise Scaling: 0.2661588570956837, Memory size: 68810 and Num Steps: 144\n",
      "Episode 1624\tAverage Score: 0.24\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2658926982385881, Memory size: 68863 and Num Steps: 53\n",
      "Episode 1625\tAverage Score: 0.24\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.26562680554034945, Memory size: 69010 and Num Steps: 147\n",
      "Episode 1626\tAverage Score: 0.25\tCurrent Score: 0.800000011920929\n",
      "Noise Scaling: 0.26536117873480913, Memory size: 69312 and Num Steps: 302\n",
      "Episode 1627\tAverage Score: 0.25\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2650958175560743, Memory size: 69339 and Num Steps: 27\n",
      "Episode 1628\tAverage Score: 0.25\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.26483072173851824, Memory size: 69432 and Num Steps: 93\n",
      "Episode 1629\tAverage Score: 0.25\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2645658910167797, Memory size: 69502 and Num Steps: 70\n",
      "Episode 1630\tAverage Score: 0.25\tCurrent Score: 0.9000000134110451\n",
      "Noise Scaling: 0.26430132512576293, Memory size: 69873 and Num Steps: 371\n",
      "Episode 1631\tAverage Score: 0.26\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.26403702380063715, Memory size: 70059 and Num Steps: 186\n",
      "Episode 1632\tAverage Score: 0.26\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.2637729867768365, Memory size: 70186 and Num Steps: 127\n",
      "Episode 1633\tAverage Score: 0.26\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.2635092137900597, Memory size: 70371 and Num Steps: 185\n",
      "Episode 1634\tAverage Score: 0.26\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2632457045762696, Memory size: 70439 and Num Steps: 68\n",
      "Episode 1635\tAverage Score: 0.26\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.26298245887169336, Memory size: 70510 and Num Steps: 71\n",
      "Episode 1636\tAverage Score: 0.26\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2627194764128217, Memory size: 70580 and Num Steps: 70\n",
      "Episode 1637\tAverage Score: 0.26\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.26245675693640885, Memory size: 70707 and Num Steps: 127\n",
      "Episode 1638\tAverage Score: 0.26\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.26219430017947243, Memory size: 70738 and Num Steps: 31\n",
      "Episode 1639\tAverage Score: 0.25\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.26193210587929294, Memory size: 70752 and Num Steps: 14\n",
      "Episode 1640\tAverage Score: 0.25\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2616701737734137, Memory size: 70829 and Num Steps: 77\n",
      "Episode 1641\tAverage Score: 0.25\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.26140850359964024, Memory size: 70919 and Num Steps: 90\n",
      "Episode 1642\tAverage Score: 0.25\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2611470950960406, Memory size: 70970 and Num Steps: 51\n",
      "Episode 1643\tAverage Score: 0.25\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.26088594800094456, Memory size: 71040 and Num Steps: 70\n",
      "Episode 1644\tAverage Score: 0.25\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.2606250620529436, Memory size: 71281 and Num Steps: 241\n",
      "Episode 1645\tAverage Score: 0.25\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2603644369908907, Memory size: 71335 and Num Steps: 54\n",
      "Episode 1646\tAverage Score: 0.25\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.2601040725538998, Memory size: 71348 and Num Steps: 13\n",
      "Episode 1647\tAverage Score: 0.25\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.2598439684813459, Memory size: 71418 and Num Steps: 70\n",
      "Episode 1648\tAverage Score: 0.25\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.25958412451286456, Memory size: 71621 and Num Steps: 203\n",
      "Episode 1649\tAverage Score: 0.25\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.2593245403883517, Memory size: 71900 and Num Steps: 279\n",
      "Episode 1650\tAverage Score: 0.25\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2590652158479633, Memory size: 71970 and Num Steps: 70\n",
      "Episode 1651\tAverage Score: 0.25\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2588061506321154, Memory size: 72041 and Num Steps: 71\n",
      "Episode 1652\tAverage Score: 0.25\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.25854734448148325, Memory size: 72186 and Num Steps: 145\n",
      "Episode 1653\tAverage Score: 0.25\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.25828879713700176, Memory size: 72309 and Num Steps: 123\n",
      "Episode 1654\tAverage Score: 0.25\tCurrent Score: 0.7900000121444464\n",
      "Noise Scaling: 0.2580305083398648, Memory size: 72606 and Num Steps: 297\n",
      "Episode 1655\tAverage Score: 0.25\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2577724778315249, Memory size: 72655 and Num Steps: 49\n",
      "Episode 1656\tAverage Score: 0.25\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.25751470535369336, Memory size: 72781 and Num Steps: 126\n",
      "Episode 1657\tAverage Score: 0.25\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2572571906483397, Memory size: 72833 and Num Steps: 52\n",
      "Episode 1658\tAverage Score: 0.25\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.25699993345769134, Memory size: 72883 and Num Steps: 50\n",
      "Episode 1659\tAverage Score: 0.25\tCurrent Score: 0.800000011920929\n",
      "Noise Scaling: 0.25674293352423366, Memory size: 73205 and Num Steps: 322\n",
      "Episode 1660\tAverage Score: 0.25\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.25648619059070943, Memory size: 73315 and Num Steps: 110\n",
      "Episode 1661\tAverage Score: 0.25\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2562297044001187, Memory size: 73367 and Num Steps: 52\n",
      "Episode 1662\tAverage Score: 0.25\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2559734746957186, Memory size: 73474 and Num Steps: 107\n",
      "Episode 1663\tAverage Score: 0.25\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.2557175012210229, Memory size: 73603 and Num Steps: 129\n",
      "Episode 1664\tAverage Score: 0.25\tCurrent Score: 0.9000000134110451\n",
      "Noise Scaling: 0.2554617837198018, Memory size: 73972 and Num Steps: 369\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1665\tAverage Score: 0.25\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.25520632193608206, Memory size: 73986 and Num Steps: 14\n",
      "Episode 1666\tAverage Score: 0.25\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.25495111561414596, Memory size: 74056 and Num Steps: 70\n",
      "Episode 1667\tAverage Score: 0.25\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.25469616449853183, Memory size: 74227 and Num Steps: 171\n",
      "Episode 1668\tAverage Score: 0.26\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.2544414683340333, Memory size: 74502 and Num Steps: 275\n",
      "Episode 1669\tAverage Score: 0.26\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.2541870268656993, Memory size: 74613 and Num Steps: 111\n",
      "Episode 1670\tAverage Score: 0.26\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2539328398388336, Memory size: 74684 and Num Steps: 71\n",
      "Episode 1671\tAverage Score: 0.27\tCurrent Score: 0.490000007674098\n",
      "Noise Scaling: 0.2536789069989947, Memory size: 74877 and Num Steps: 193\n",
      "Episode 1672\tAverage Score: 0.27\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.25342522809199575, Memory size: 75081 and Num Steps: 204\n",
      "Episode 1673\tAverage Score: 0.27\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2531718028639037, Memory size: 75171 and Num Steps: 90\n",
      "Episode 1674\tAverage Score: 0.27\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.25291863106103984, Memory size: 75315 and Num Steps: 144\n",
      "Episode 1675\tAverage Score: 0.27\tCurrent Score: 0.2900000065565109\n",
      "Noise Scaling: 0.25266571242997876, Memory size: 75422 and Num Steps: 107\n",
      "Episode 1676\tAverage Score: 0.27\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.25241304671754883, Memory size: 75474 and Num Steps: 52\n",
      "Episode 1677\tAverage Score: 0.27\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.25216063367083125, Memory size: 75525 and Num Steps: 51\n",
      "Episode 1678\tAverage Score: 0.27\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2519084730371604, Memory size: 75596 and Num Steps: 71\n",
      "Episode 1679\tAverage Score: 0.27\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2516565645641233, Memory size: 75626 and Num Steps: 30\n",
      "Episode 1680\tAverage Score: 0.27\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.25140490799955917, Memory size: 75682 and Num Steps: 56\n",
      "Episode 1681\tAverage Score: 0.27\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.2511535030915596, Memory size: 75925 and Num Steps: 243\n",
      "Episode 1682\tAverage Score: 0.27\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.250902349588468, Memory size: 76016 and Num Steps: 91\n",
      "Episode 1683\tAverage Score: 0.27\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.25065144723887955, Memory size: 76046 and Num Steps: 30\n",
      "Episode 1684\tAverage Score: 0.27\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.25040079579164065, Memory size: 76099 and Num Steps: 53\n",
      "Episode 1685\tAverage Score: 0.26\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.250150394995849, Memory size: 76130 and Num Steps: 31\n",
      "Episode 1686\tAverage Score: 0.26\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2499002446008532, Memory size: 76183 and Num Steps: 53\n",
      "Episode 1687\tAverage Score: 0.27\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.24965034435625233, Memory size: 76462 and Num Steps: 279\n",
      "Episode 1688\tAverage Score: 0.27\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.24940069401189607, Memory size: 76534 and Num Steps: 72\n",
      "Episode 1689\tAverage Score: 0.27\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.2491512933178842, Memory size: 76603 and Num Steps: 69\n",
      "Episode 1690\tAverage Score: 0.27\tCurrent Score: 0.3900000061839819\n",
      "Noise Scaling: 0.2489021420245663, Memory size: 76747 and Num Steps: 144\n",
      "Episode 1691\tAverage Score: 0.27\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.24865323988254173, Memory size: 76822 and Num Steps: 75\n",
      "Episode 1692\tAverage Score: 0.27\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.2484045866426592, Memory size: 77027 and Num Steps: 205\n",
      "Episode 1693\tAverage Score: 0.28\tCurrent Score: 1.0000000149011612\n",
      "Noise Scaling: 0.24815618205601653, Memory size: 77426 and Num Steps: 399\n",
      "Episode 1694\tAverage Score: 0.29\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.24790802587396052, Memory size: 77686 and Num Steps: 260\n",
      "Episode 1695\tAverage Score: 0.29\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.24766011784808656, Memory size: 77966 and Num Steps: 280\n",
      "Episode 1696\tAverage Score: 0.29\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.24741245773023846, Memory size: 78029 and Num Steps: 63\n",
      "Episode 1697\tAverage Score: 0.29\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.24716504527250824, Memory size: 78253 and Num Steps: 224\n",
      "Episode 1698\tAverage Score: 0.29\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.2469178802272357, Memory size: 78416 and Num Steps: 163\n",
      "Episode 1699\tAverage Score: 0.29\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.24667096234700847, Memory size: 78484 and Num Steps: 68\n",
      "Episode 1700\tAverage Score: 0.29\tCurrent Score: 0.490000007674098\n",
      "Noise Scaling: 0.2464242913846615, Memory size: 78666 and Num Steps: 182\n",
      "Episode 1701\tAverage Score: 0.30\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.24617786709327683, Memory size: 78877 and Num Steps: 211\n",
      "Episode 1702\tAverage Score: 0.30\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.24593168922618353, Memory size: 79059 and Num Steps: 182\n",
      "Episode 1703\tAverage Score: 0.30\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.24568575753695734, Memory size: 79238 and Num Steps: 179\n",
      "Episode 1704\tAverage Score: 0.30\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2454400717794204, Memory size: 79289 and Num Steps: 51\n",
      "Episode 1705\tAverage Score: 0.31\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.24519463170764097, Memory size: 79435 and Num Steps: 146\n",
      "Episode 1706\tAverage Score: 0.31\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.24494943707593334, Memory size: 79696 and Num Steps: 261\n",
      "Episode 1707\tAverage Score: 0.32\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.2447044876388574, Memory size: 79977 and Num Steps: 281\n",
      "Episode 1708\tAverage Score: 0.32\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.24445978315121855, Memory size: 80064 and Num Steps: 87\n",
      "Episode 1709\tAverage Score: 0.31\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.2442153233680673, Memory size: 80134 and Num Steps: 70\n",
      "Episode 1710\tAverage Score: 0.31\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.24397110804469926, Memory size: 80204 and Num Steps: 70\n",
      "Episode 1711\tAverage Score: 0.31\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.24372713693665454, Memory size: 80436 and Num Steps: 232\n",
      "Episode 1712\tAverage Score: 0.32\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2434834097997179, Memory size: 80505 and Num Steps: 69\n",
      "Episode 1713\tAverage Score: 0.32\tCurrent Score: 0.2900000046938658\n",
      "Noise Scaling: 0.24323992638991818, Memory size: 80608 and Num Steps: 103\n",
      "Episode 1714\tAverage Score: 0.32\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.24299668646352826, Memory size: 80769 and Num Steps: 161\n",
      "Episode 1715\tAverage Score: 0.32\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.24275368977706474, Memory size: 80837 and Num Steps: 68\n",
      "Episode 1716\tAverage Score: 0.32\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.24251093608728766, Memory size: 80947 and Num Steps: 110\n",
      "Episode 1717\tAverage Score: 0.33\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.2422684251512004, Memory size: 81169 and Num Steps: 222\n",
      "Episode 1718\tAverage Score: 0.33\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.24202615672604919, Memory size: 81355 and Num Steps: 186\n",
      "Episode 1719\tAverage Score: 0.33\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.24178413056932313, Memory size: 81404 and Num Steps: 49\n",
      "Episode 1720\tAverage Score: 0.33\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.2415423464387538, Memory size: 81589 and Num Steps: 185\n",
      "Episode 1721\tAverage Score: 0.33\tCurrent Score: 0.09000000357627869\n",
      "Noise Scaling: 0.24130080409231505, Memory size: 81620 and Num Steps: 31\n",
      "Episode 1722\tAverage Score: 0.33\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.24105950328822273, Memory size: 81710 and Num Steps: 90\n",
      "Episode 1723\tAverage Score: 0.33\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.24081844378493453, Memory size: 81859 and Num Steps: 149\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1724\tAverage Score: 0.33\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.24057762534114957, Memory size: 81948 and Num Steps: 89\n",
      "Episode 1725\tAverage Score: 0.33\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.24033704771580844, Memory size: 82171 and Num Steps: 223\n",
      "Episode 1726\tAverage Score: 0.32\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.24009671066809263, Memory size: 82201 and Num Steps: 30\n",
      "Episode 1727\tAverage Score: 0.33\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.23985661395742452, Memory size: 82365 and Num Steps: 164\n",
      "Episode 1728\tAverage Score: 0.33\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.2396167573434671, Memory size: 82488 and Num Steps: 123\n",
      "Episode 1729\tAverage Score: 0.33\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.23937714058612364, Memory size: 82558 and Num Steps: 70\n",
      "Episode 1730\tAverage Score: 0.32\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.23913776344553753, Memory size: 82752 and Num Steps: 194\n",
      "Episode 1731\tAverage Score: 0.33\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.23889862568209197, Memory size: 83017 and Num Steps: 265\n",
      "Episode 1732\tAverage Score: 0.32\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2386597270564099, Memory size: 83047 and Num Steps: 30\n",
      "Episode 1733\tAverage Score: 0.32\tCurrent Score: 0.490000007674098\n",
      "Noise Scaling: 0.23842106732935348, Memory size: 83230 and Num Steps: 183\n",
      "Episode 1734\tAverage Score: 0.32\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.23818264626202412, Memory size: 83243 and Num Steps: 13\n",
      "Episode 1735\tAverage Score: 0.32\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.2379444636157621, Memory size: 83392 and Num Steps: 149\n",
      "Episode 1736\tAverage Score: 0.32\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.23770651915214633, Memory size: 83459 and Num Steps: 67\n",
      "Episode 1737\tAverage Score: 0.32\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.2374688126329942, Memory size: 83592 and Num Steps: 133\n",
      "Episode 1738\tAverage Score: 0.32\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2372313438203612, Memory size: 83643 and Num Steps: 51\n",
      "Episode 1739\tAverage Score: 0.33\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.23699411247654084, Memory size: 83693 and Num Steps: 50\n",
      "Episode 1740\tAverage Score: 0.34\tCurrent Score: 1.5000000223517418\n",
      "Noise Scaling: 0.2367571183640643, Memory size: 84259 and Num Steps: 566\n",
      "Episode 1741\tAverage Score: 0.34\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.23652036124570022, Memory size: 84310 and Num Steps: 51\n",
      "Episode 1742\tAverage Score: 0.34\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.23628384088445453, Memory size: 84590 and Num Steps: 280\n",
      "Episode 1743\tAverage Score: 0.34\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.23604755704357008, Memory size: 84647 and Num Steps: 57\n",
      "Episode 1744\tAverage Score: 0.34\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2358115094865265, Memory size: 84717 and Num Steps: 70\n",
      "Episode 1745\tAverage Score: 0.34\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.23557569797703998, Memory size: 84826 and Num Steps: 109\n",
      "Episode 1746\tAverage Score: 0.35\tCurrent Score: 0.800000011920929\n",
      "Noise Scaling: 0.23534012227906292, Memory size: 85140 and Num Steps: 314\n",
      "Episode 1747\tAverage Score: 0.35\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.23510478215678388, Memory size: 85192 and Num Steps: 52\n",
      "Episode 1748\tAverage Score: 0.35\tCurrent Score: 0.3900000061839819\n",
      "Noise Scaling: 0.23486967737462708, Memory size: 85335 and Num Steps: 143\n",
      "Episode 1749\tAverage Score: 0.34\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.23463480769725245, Memory size: 85402 and Num Steps: 67\n",
      "Episode 1750\tAverage Score: 0.35\tCurrent Score: 1.5000000223517418\n",
      "Noise Scaling: 0.2344001728895552, Memory size: 85969 and Num Steps: 567\n",
      "Episode 1751\tAverage Score: 0.35\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.23416577271666567, Memory size: 86038 and Num Steps: 69\n",
      "Episode 1752\tAverage Score: 0.35\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.233931606943949, Memory size: 86143 and Num Steps: 105\n",
      "Episode 1753\tAverage Score: 0.35\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.23369767533700503, Memory size: 86233 and Num Steps: 90\n",
      "Episode 1754\tAverage Score: 0.35\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.23346397766166804, Memory size: 86322 and Num Steps: 89\n",
      "Episode 1755\tAverage Score: 0.35\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.23323051368400635, Memory size: 86507 and Num Steps: 185\n",
      "Episode 1756\tAverage Score: 0.35\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.23299728317032237, Memory size: 86673 and Num Steps: 166\n",
      "Episode 1757\tAverage Score: 0.35\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.23276428588715203, Memory size: 86764 and Num Steps: 91\n",
      "Episode 1758\tAverage Score: 0.36\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.2325315216012649, Memory size: 86985 and Num Steps: 221\n",
      "Episode 1759\tAverage Score: 0.36\tCurrent Score: 0.800000011920929\n",
      "Noise Scaling: 0.23229899007966362, Memory size: 87297 and Num Steps: 312\n",
      "Episode 1760\tAverage Score: 0.35\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.23206669108958394, Memory size: 87349 and Num Steps: 52\n",
      "Episode 1761\tAverage Score: 0.36\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.23183462439849437, Memory size: 87557 and Num Steps: 208\n",
      "Episode 1762\tAverage Score: 0.36\tCurrent Score: 0.490000007674098\n",
      "Noise Scaling: 0.23160278977409587, Memory size: 87756 and Num Steps: 199\n",
      "Episode 1763\tAverage Score: 0.36\tCurrent Score: 0.490000007674098\n",
      "Noise Scaling: 0.23137118698432177, Memory size: 87940 and Num Steps: 184\n",
      "Episode 1764\tAverage Score: 0.36\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.23113981579733744, Memory size: 88007 and Num Steps: 67\n",
      "Episode 1765\tAverage Score: 0.36\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.2309086759815401, Memory size: 88137 and Num Steps: 130\n",
      "Episode 1766\tAverage Score: 0.36\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.23067776730555858, Memory size: 88227 and Num Steps: 90\n",
      "Episode 1767\tAverage Score: 0.37\tCurrent Score: 0.9900000169873238\n",
      "Noise Scaling: 0.23044708953825302, Memory size: 88602 and Num Steps: 375\n",
      "Episode 1768\tAverage Score: 0.36\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.23021664244871476, Memory size: 88670 and Num Steps: 68\n",
      "Episode 1769\tAverage Score: 0.36\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.22998642580626605, Memory size: 88831 and Num Steps: 161\n",
      "Episode 1770\tAverage Score: 0.36\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.22975643938045978, Memory size: 88861 and Num Steps: 30\n",
      "Episode 1771\tAverage Score: 0.36\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.22952668294107934, Memory size: 88910 and Num Steps: 49\n",
      "Episode 1772\tAverage Score: 0.35\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.22929715625813823, Memory size: 88959 and Num Steps: 49\n",
      "Episode 1773\tAverage Score: 0.35\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2290678591018801, Memory size: 88989 and Num Steps: 30\n",
      "Episode 1774\tAverage Score: 0.35\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.22883879124277823, Memory size: 89058 and Num Steps: 69\n",
      "Episode 1775\tAverage Score: 0.35\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.22860995245153545, Memory size: 89310 and Num Steps: 252\n",
      "Episode 1776\tAverage Score: 0.35\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.2283813424990839, Memory size: 89324 and Num Steps: 14\n",
      "Episode 1777\tAverage Score: 0.35\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.22815296115658482, Memory size: 89339 and Num Steps: 15\n",
      "Episode 1778\tAverage Score: 0.35\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.22792480819542824, Memory size: 89362 and Num Steps: 23\n",
      "Episode 1779\tAverage Score: 0.35\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.22769688338723282, Memory size: 89414 and Num Steps: 52\n",
      "Episode 1780\tAverage Score: 0.36\tCurrent Score: 0.490000007674098\n",
      "Noise Scaling: 0.2274691865038456, Memory size: 89630 and Num Steps: 216\n",
      "Episode 1781\tAverage Score: 0.36\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.22724171731734175, Memory size: 89894 and Num Steps: 264\n",
      "Episode 1782\tAverage Score: 0.36\tCurrent Score: 0.490000007674098\n",
      "Noise Scaling: 0.22701447560002438, Memory size: 90093 and Num Steps: 199\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1783\tAverage Score: 0.37\tCurrent Score: 0.800000011920929\n",
      "Noise Scaling: 0.22678746112442438, Memory size: 90411 and Num Steps: 318\n",
      "Episode 1784\tAverage Score: 0.37\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.22656067366329993, Memory size: 90481 and Num Steps: 70\n",
      "Episode 1785\tAverage Score: 0.37\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.22633411298963665, Memory size: 90729 and Num Steps: 248\n",
      "Episode 1786\tAverage Score: 0.37\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.22610777887664701, Memory size: 90781 and Num Steps: 52\n",
      "Episode 1787\tAverage Score: 0.37\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.22588167109777035, Memory size: 91010 and Num Steps: 229\n",
      "Episode 1788\tAverage Score: 0.38\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.22565578942667258, Memory size: 91231 and Num Steps: 221\n",
      "Episode 1789\tAverage Score: 0.38\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.22543013363724593, Memory size: 91298 and Num Steps: 67\n",
      "Episode 1790\tAverage Score: 0.37\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.22520470350360866, Memory size: 91332 and Num Steps: 34\n",
      "Episode 1791\tAverage Score: 0.37\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.22497949880010507, Memory size: 91423 and Num Steps: 91\n",
      "Episode 1792\tAverage Score: 0.37\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.22475451930130497, Memory size: 91583 and Num Steps: 160\n",
      "Episode 1793\tAverage Score: 0.36\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.22452976478200365, Memory size: 91659 and Num Steps: 76\n",
      "Episode 1794\tAverage Score: 0.36\tCurrent Score: 0.9000000134110451\n",
      "Noise Scaling: 0.22430523501722166, Memory size: 92025 and Num Steps: 366\n",
      "Episode 1795\tAverage Score: 0.36\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.22408092978220442, Memory size: 92093 and Num Steps: 68\n",
      "Episode 1796\tAverage Score: 0.38\tCurrent Score: 2.500000037252903\n",
      "Noise Scaling: 0.2238568488524222, Memory size: 93093 and Num Steps: 1000\n",
      "Episode 1797\tAverage Score: 0.39\tCurrent Score: 0.9000000134110451\n",
      "Noise Scaling: 0.22363299200356979, Memory size: 93444 and Num Steps: 351\n",
      "Episode 1798\tAverage Score: 0.38\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.22340935901156622, Memory size: 93517 and Num Steps: 73\n",
      "Episode 1799\tAverage Score: 0.38\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.22318594965255467, Memory size: 93531 and Num Steps: 14\n",
      "Episode 1800\tAverage Score: 0.39\tCurrent Score: 1.2000000178813934\n",
      "Noise Scaling: 0.2229627637029021, Memory size: 93983 and Num Steps: 452\n",
      "Episode 1801\tAverage Score: 0.39\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.2227398009391992, Memory size: 94089 and Num Steps: 106\n",
      "Episode 1802\tAverage Score: 0.38\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.22251706113826, Memory size: 94178 and Num Steps: 89\n",
      "Episode 1803\tAverage Score: 0.38\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.22229454407712174, Memory size: 94325 and Num Steps: 147\n",
      "Episode 1804\tAverage Score: 0.38\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.22207224953304464, Memory size: 94394 and Num Steps: 69\n",
      "Episode 1805\tAverage Score: 0.40\tCurrent Score: 1.8000000268220901\n",
      "Noise Scaling: 0.22185017728351158, Memory size: 95095 and Num Steps: 701\n",
      "Episode 1806\tAverage Score: 0.40\tCurrent Score: 0.9000000134110451\n",
      "Noise Scaling: 0.22162832710622807, Memory size: 95462 and Num Steps: 367\n",
      "Episode 1807\tAverage Score: 0.40\tCurrent Score: 0.490000007674098\n",
      "Noise Scaling: 0.22140669877912184, Memory size: 95641 and Num Steps: 179\n",
      "Episode 1808\tAverage Score: 0.40\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2211852920803427, Memory size: 95708 and Num Steps: 67\n",
      "Episode 1809\tAverage Score: 0.41\tCurrent Score: 1.1000000163912773\n",
      "Noise Scaling: 0.22096410678826237, Memory size: 96139 and Num Steps: 431\n",
      "Episode 1810\tAverage Score: 0.41\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2207431426814741, Memory size: 96209 and Num Steps: 70\n",
      "Episode 1811\tAverage Score: 0.41\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.22052239953879263, Memory size: 96374 and Num Steps: 165\n",
      "Episode 1812\tAverage Score: 0.41\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.22030187713925384, Memory size: 96575 and Num Steps: 201\n",
      "Episode 1813\tAverage Score: 0.43\tCurrent Score: 2.500000037252903\n",
      "Noise Scaling: 0.22008157526211458, Memory size: 97520 and Num Steps: 945\n",
      "Episode 1814\tAverage Score: 0.44\tCurrent Score: 1.8000000268220901\n",
      "Noise Scaling: 0.21986149368685248, Memory size: 98240 and Num Steps: 720\n",
      "Episode 1815\tAverage Score: 0.44\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.21964163219316563, Memory size: 98345 and Num Steps: 105\n",
      "Episode 1816\tAverage Score: 0.45\tCurrent Score: 0.800000011920929\n",
      "Noise Scaling: 0.21942199056097245, Memory size: 98655 and Num Steps: 310\n",
      "Episode 1817\tAverage Score: 0.45\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.21920256857041148, Memory size: 98700 and Num Steps: 45\n",
      "Episode 1818\tAverage Score: 0.44\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.21898336600184107, Memory size: 98769 and Num Steps: 69\n",
      "Episode 1819\tAverage Score: 0.44\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.21876438263583922, Memory size: 98821 and Num Steps: 52\n",
      "Episode 1820\tAverage Score: 0.44\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.2185456182532034, Memory size: 98836 and Num Steps: 15\n",
      "Episode 1821\tAverage Score: 0.44\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2183270726349502, Memory size: 98929 and Num Steps: 93\n",
      "Episode 1822\tAverage Score: 0.44\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.21810874556231524, Memory size: 98977 and Num Steps: 48\n",
      "Episode 1823\tAverage Score: 0.43\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.21789063681675291, Memory size: 99029 and Num Steps: 52\n",
      "Episode 1824\tAverage Score: 0.45\tCurrent Score: 1.3000000193715096\n",
      "Noise Scaling: 0.21767274617993618, Memory size: 99538 and Num Steps: 509\n",
      "Episode 1825\tAverage Score: 0.44\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.21745507343375622, Memory size: 99590 and Num Steps: 52\n",
      "Episode 1826\tAverage Score: 0.45\tCurrent Score: 0.9000000134110451\n",
      "Noise Scaling: 0.2172376183603225, Memory size: 99934 and Num Steps: 344\n",
      "Episode 1827\tAverage Score: 0.44\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.21702038074196214, Memory size: 99948 and Num Steps: 14\n",
      "Episode 1828\tAverage Score: 0.44\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2168033603612202, Memory size: 99978 and Num Steps: 30\n",
      "Episode 1829\tAverage Score: 0.44\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.21658655700085896, Memory size: 100000 and Num Steps: 52\n",
      "Episode 1830\tAverage Score: 0.45\tCurrent Score: 1.2000000178813934\n",
      "Noise Scaling: 0.2163699704438581, Memory size: 100000 and Num Steps: 452\n",
      "Episode 1831\tAverage Score: 0.44\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.21615360047341425, Memory size: 100000 and Num Steps: 68\n",
      "Episode 1832\tAverage Score: 0.45\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.21593744687294084, Memory size: 100000 and Num Steps: 293\n",
      "Episode 1833\tAverage Score: 0.45\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.2157215094260679, Memory size: 100000 and Num Steps: 279\n",
      "Episode 1834\tAverage Score: 0.45\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.21550578791664182, Memory size: 100000 and Num Steps: 72\n",
      "Episode 1835\tAverage Score: 0.45\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.2152902821287252, Memory size: 100000 and Num Steps: 14\n",
      "Episode 1836\tAverage Score: 0.45\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.21507499184659645, Memory size: 100000 and Num Steps: 85\n",
      "Episode 1837\tAverage Score: 0.45\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.21485991685474987, Memory size: 100000 and Num Steps: 65\n",
      "Episode 1838\tAverage Score: 0.45\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.21464505693789512, Memory size: 100000 and Num Steps: 281\n",
      "Episode 1839\tAverage Score: 0.46\tCurrent Score: 0.2900000046938658\n",
      "Noise Scaling: 0.2144304118809572, Memory size: 100000 and Num Steps: 104\n",
      "Episode 1840\tAverage Score: 0.44\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.21421598146907628, Memory size: 100000 and Num Steps: 166\n",
      "Episode 1841\tAverage Score: 0.44\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2140017654876072, Memory size: 100000 and Num Steps: 54\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1842\tAverage Score: 0.44\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.21378776372211958, Memory size: 100000 and Num Steps: 290\n",
      "Episode 1843\tAverage Score: 0.44\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.21357397595839747, Memory size: 100000 and Num Steps: 52\n",
      "Episode 1844\tAverage Score: 0.44\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.21336040198243905, Memory size: 100000 and Num Steps: 30\n",
      "Episode 1845\tAverage Score: 0.45\tCurrent Score: 1.0000000149011612\n",
      "Noise Scaling: 0.21314704158045664, Memory size: 100000 and Num Steps: 372\n",
      "Episode 1846\tAverage Score: 0.45\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.21293389453887618, Memory size: 100000 and Num Steps: 224\n",
      "Episode 1847\tAverage Score: 0.46\tCurrent Score: 1.5000000223517418\n",
      "Noise Scaling: 0.2127209606443373, Memory size: 100000 and Num Steps: 567\n",
      "Episode 1848\tAverage Score: 0.46\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.21250823968369295, Memory size: 100000 and Num Steps: 65\n",
      "Episode 1849\tAverage Score: 0.47\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.21229573144400926, Memory size: 100000 and Num Steps: 259\n",
      "Episode 1850\tAverage Score: 0.46\tCurrent Score: 1.4000000208616257\n",
      "Noise Scaling: 0.21208343571256524, Memory size: 100000 and Num Steps: 542\n",
      "Episode 1851\tAverage Score: 0.46\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.2118713522768527, Memory size: 100000 and Num Steps: 14\n",
      "Episode 1852\tAverage Score: 0.46\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.21165948092457584, Memory size: 100000 and Num Steps: 109\n",
      "Episode 1853\tAverage Score: 0.46\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.21144782144365126, Memory size: 100000 and Num Steps: 66\n",
      "Episode 1854\tAverage Score: 0.46\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2112363736222076, Memory size: 100000 and Num Steps: 30\n",
      "Episode 1855\tAverage Score: 0.46\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.2110251372485854, Memory size: 100000 and Num Steps: 105\n",
      "Episode 1856\tAverage Score: 0.47\tCurrent Score: 1.4000000208616257\n",
      "Noise Scaling: 0.2108141121113368, Memory size: 100000 and Num Steps: 528\n",
      "Episode 1857\tAverage Score: 0.47\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.21060329799922548, Memory size: 100000 and Num Steps: 148\n",
      "Episode 1858\tAverage Score: 0.47\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.21039269470122623, Memory size: 100000 and Num Steps: 29\n",
      "Episode 1859\tAverage Score: 0.46\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.21018230200652502, Memory size: 100000 and Num Steps: 184\n",
      "Episode 1860\tAverage Score: 0.46\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.20997211970451848, Memory size: 100000 and Num Steps: 68\n",
      "Episode 1861\tAverage Score: 0.46\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.20976214758481398, Memory size: 100000 and Num Steps: 65\n",
      "Episode 1862\tAverage Score: 0.46\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.20955238543722918, Memory size: 100000 and Num Steps: 223\n",
      "Episode 1863\tAverage Score: 0.46\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.20934283305179194, Memory size: 100000 and Num Steps: 281\n",
      "Episode 1864\tAverage Score: 0.46\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.20913349021874014, Memory size: 100000 and Num Steps: 70\n",
      "Episode 1865\tAverage Score: 0.46\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.20892435672852142, Memory size: 100000 and Num Steps: 146\n",
      "Episode 1866\tAverage Score: 0.46\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.20871543237179288, Memory size: 100000 and Num Steps: 30\n",
      "Episode 1867\tAverage Score: 0.45\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.20850671693942108, Memory size: 100000 and Num Steps: 70\n",
      "Episode 1868\tAverage Score: 0.46\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.20829821022248166, Memory size: 100000 and Num Steps: 223\n",
      "Episode 1869\tAverage Score: 0.46\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.20808991201225918, Memory size: 100000 and Num Steps: 123\n",
      "Episode 1870\tAverage Score: 0.46\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.20788182210024692, Memory size: 100000 and Num Steps: 267\n",
      "Episode 1871\tAverage Score: 0.46\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.20767394027814667, Memory size: 100000 and Num Steps: 67\n",
      "Episode 1872\tAverage Score: 0.46\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.20746626633786852, Memory size: 100000 and Num Steps: 68\n",
      "Episode 1873\tAverage Score: 0.46\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.20725880007153066, Memory size: 100000 and Num Steps: 65\n",
      "Episode 1874\tAverage Score: 0.46\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.20705154127145914, Memory size: 100000 and Num Steps: 30\n",
      "Episode 1875\tAverage Score: 0.46\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.20684448973018768, Memory size: 100000 and Num Steps: 122\n",
      "Episode 1876\tAverage Score: 0.46\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.2066376452404575, Memory size: 100000 and Num Steps: 226\n",
      "Episode 1877\tAverage Score: 0.47\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.20643100759521701, Memory size: 100000 and Num Steps: 30\n",
      "Episode 1878\tAverage Score: 0.47\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2062245765876218, Memory size: 100000 and Num Steps: 49\n",
      "Episode 1879\tAverage Score: 0.47\tCurrent Score: 0.3900000061839819\n",
      "Noise Scaling: 0.2060183520110342, Memory size: 100000 and Num Steps: 145\n",
      "Episode 1880\tAverage Score: 0.46\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.20581233365902316, Memory size: 100000 and Num Steps: 30\n",
      "Episode 1881\tAverage Score: 0.46\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.20560652132536414, Memory size: 100000 and Num Steps: 30\n",
      "Episode 1882\tAverage Score: 0.45\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.20540091480403877, Memory size: 100000 and Num Steps: 30\n",
      "Episode 1883\tAverage Score: 0.45\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.20519551388923474, Memory size: 100000 and Num Steps: 87\n",
      "Episode 1884\tAverage Score: 0.45\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.20499031837534548, Memory size: 100000 and Num Steps: 109\n",
      "Episode 1885\tAverage Score: 0.46\tCurrent Score: 1.600000023841858\n",
      "Noise Scaling: 0.20478532805697014, Memory size: 100000 and Num Steps: 603\n",
      "Episode 1886\tAverage Score: 0.46\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.20458054272891318, Memory size: 100000 and Num Steps: 88\n",
      "Episode 1887\tAverage Score: 0.46\tCurrent Score: 1.1000000163912773\n",
      "Noise Scaling: 0.20437596218618426, Memory size: 100000 and Num Steps: 410\n",
      "Episode 1888\tAverage Score: 0.46\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.20417158622399809, Memory size: 100000 and Num Steps: 32\n",
      "Episode 1889\tAverage Score: 0.46\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.20396741463777407, Memory size: 100000 and Num Steps: 214\n",
      "Episode 1890\tAverage Score: 0.47\tCurrent Score: 1.1000000163912773\n",
      "Noise Scaling: 0.2037634472231363, Memory size: 100000 and Num Steps: 435\n",
      "Episode 1891\tAverage Score: 0.47\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.20355968377591316, Memory size: 100000 and Num Steps: 53\n",
      "Episode 1892\tAverage Score: 0.47\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.20335612409213724, Memory size: 100000 and Num Steps: 32\n",
      "Episode 1893\tAverage Score: 0.47\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.20315276796804513, Memory size: 100000 and Num Steps: 106\n",
      "Episode 1894\tAverage Score: 0.46\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.20294961520007707, Memory size: 100000 and Num Steps: 34\n",
      "Episode 1895\tAverage Score: 0.46\tCurrent Score: 0.2900000046938658\n",
      "Noise Scaling: 0.202746665584877, Memory size: 100000 and Num Steps: 106\n",
      "Episode 1896\tAverage Score: 0.44\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.20254391891929213, Memory size: 100000 and Num Steps: 141\n",
      "Episode 1897\tAverage Score: 0.44\tCurrent Score: 0.490000007674098\n",
      "Noise Scaling: 0.20234137500037283, Memory size: 100000 and Num Steps: 183\n",
      "Episode 1898\tAverage Score: 0.44\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.20213903362537244, Memory size: 100000 and Num Steps: 106\n",
      "Episode 1899\tAverage Score: 0.44\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.20193689459174707, Memory size: 100000 and Num Steps: 14\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1900\tAverage Score: 0.44\tCurrent Score: 1.700000025331974\n",
      "Noise Scaling: 0.20173495769715533, Memory size: 100000 and Num Steps: 687\n",
      "Episode 1901\tAverage Score: 0.44\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.20153322273945817, Memory size: 100000 and Num Steps: 69\n",
      "Episode 1902\tAverage Score: 0.44\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.20133168951671873, Memory size: 100000 and Num Steps: 69\n",
      "Episode 1903\tAverage Score: 0.45\tCurrent Score: 0.800000011920929\n",
      "Noise Scaling: 0.201130357827202, Memory size: 100000 and Num Steps: 327\n",
      "Episode 1904\tAverage Score: 0.45\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.2009292274693748, Memory size: 100000 and Num Steps: 277\n",
      "Episode 1905\tAverage Score: 0.43\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.2007282982419054, Memory size: 100000 and Num Steps: 30\n",
      "Episode 1906\tAverage Score: 0.43\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.20052756994366353, Memory size: 100000 and Num Steps: 30\n",
      "Episode 1907\tAverage Score: 0.42\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.20032704237371984, Memory size: 100000 and Num Steps: 51\n",
      "Episode 1908\tAverage Score: 0.42\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.20012671533134613, Memory size: 100000 and Num Steps: 30\n",
      "Episode 1909\tAverage Score: 0.41\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.19992658861601478, Memory size: 100000 and Num Steps: 91\n",
      "Episode 1910\tAverage Score: 0.43\tCurrent Score: 1.600000023841858\n",
      "Noise Scaling: 0.19972666202739878, Memory size: 100000 and Num Steps: 621\n",
      "Episode 1911\tAverage Score: 0.42\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.19952693536537136, Memory size: 100000 and Num Steps: 13\n",
      "Episode 1912\tAverage Score: 0.42\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.19932740843000601, Memory size: 100000 and Num Steps: 72\n",
      "Episode 1913\tAverage Score: 0.39\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.199128081021576, Memory size: 100000 and Num Steps: 14\n",
      "Episode 1914\tAverage Score: 0.38\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.19892895294055443, Memory size: 100000 and Num Steps: 64\n",
      "Episode 1915\tAverage Score: 0.39\tCurrent Score: 1.0000000149011612\n",
      "Noise Scaling: 0.19873002398761386, Memory size: 100000 and Num Steps: 376\n",
      "Episode 1916\tAverage Score: 0.38\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.19853129396362626, Memory size: 100000 and Num Steps: 31\n",
      "Episode 1917\tAverage Score: 0.38\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.19833276266966263, Memory size: 100000 and Num Steps: 103\n",
      "Episode 1918\tAverage Score: 0.38\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.19813442990699295, Memory size: 100000 and Num Steps: 66\n",
      "Episode 1919\tAverage Score: 0.40\tCurrent Score: 1.700000025331974\n",
      "Noise Scaling: 0.19793629547708597, Memory size: 100000 and Num Steps: 654\n",
      "Episode 1920\tAverage Score: 0.40\tCurrent Score: 0.6900000106543303\n",
      "Noise Scaling: 0.19773835918160887, Memory size: 100000 and Num Steps: 258\n",
      "Episode 1921\tAverage Score: 0.40\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.19754062082242727, Memory size: 100000 and Num Steps: 30\n",
      "Episode 1922\tAverage Score: 0.40\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.19734308020160485, Memory size: 100000 and Num Steps: 51\n",
      "Episode 1923\tAverage Score: 0.40\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.19714573712140324, Memory size: 100000 and Num Steps: 30\n",
      "Episode 1924\tAverage Score: 0.39\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.19694859138428183, Memory size: 100000 and Num Steps: 108\n",
      "Episode 1925\tAverage Score: 0.40\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.19675164279289756, Memory size: 100000 and Num Steps: 220\n",
      "Episode 1926\tAverage Score: 0.40\tCurrent Score: 1.3900000229477882\n",
      "Noise Scaling: 0.19655489115010466, Memory size: 100000 and Num Steps: 544\n",
      "Episode 1927\tAverage Score: 0.40\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.19635833625895455, Memory size: 100000 and Num Steps: 30\n",
      "Episode 1928\tAverage Score: 0.40\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.1961619779226956, Memory size: 100000 and Num Steps: 30\n",
      "Episode 1929\tAverage Score: 0.40\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.19596581594477291, Memory size: 100000 and Num Steps: 70\n",
      "Episode 1930\tAverage Score: 0.40\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.19576985012882814, Memory size: 100000 and Num Steps: 181\n",
      "Episode 1931\tAverage Score: 0.41\tCurrent Score: 1.3000000193715096\n",
      "Noise Scaling: 0.1955740802786993, Memory size: 100000 and Num Steps: 491\n",
      "Episode 1932\tAverage Score: 0.40\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.1953785061984206, Memory size: 100000 and Num Steps: 105\n",
      "Episode 1933\tAverage Score: 0.41\tCurrent Score: 1.9000000283122063\n",
      "Noise Scaling: 0.19518312769222218, Memory size: 100000 and Num Steps: 718\n",
      "Episode 1934\tAverage Score: 0.42\tCurrent Score: 1.0000000149011612\n",
      "Noise Scaling: 0.19498794456452995, Memory size: 100000 and Num Steps: 398\n",
      "Episode 1935\tAverage Score: 0.45\tCurrent Score: 2.600000038743019\n",
      "Noise Scaling: 0.19479295661996543, Memory size: 100000 and Num Steps: 1000\n",
      "Episode 1936\tAverage Score: 0.45\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.19459816366334545, Memory size: 100000 and Num Steps: 67\n",
      "Episode 1937\tAverage Score: 0.45\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.19440356549968213, Memory size: 100000 and Num Steps: 66\n",
      "Episode 1938\tAverage Score: 0.45\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.19420916193418244, Memory size: 100000 and Num Steps: 184\n",
      "Episode 1939\tAverage Score: 0.45\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.19401495277224826, Memory size: 100000 and Num Steps: 67\n",
      "Episode 1940\tAverage Score: 0.45\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.193820937819476, Memory size: 100000 and Num Steps: 136\n",
      "Episode 1941\tAverage Score: 0.46\tCurrent Score: 1.9000000283122063\n",
      "Noise Scaling: 0.19362711688165654, Memory size: 100000 and Num Steps: 717\n",
      "Episode 1942\tAverage Score: 0.47\tCurrent Score: 0.9000000134110451\n",
      "Noise Scaling: 0.19343348976477487, Memory size: 100000 and Num Steps: 341\n",
      "Episode 1943\tAverage Score: 0.47\tCurrent Score: 1.0000000149011612\n",
      "Noise Scaling: 0.1932400562750101, Memory size: 100000 and Num Steps: 391\n",
      "Episode 1944\tAverage Score: 0.48\tCurrent Score: 1.0000000149011612\n",
      "Noise Scaling: 0.1930468162187351, Memory size: 100000 and Num Steps: 406\n",
      "Episode 1945\tAverage Score: 0.48\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.19285376940251636, Memory size: 100000 and Num Steps: 95\n",
      "Episode 1946\tAverage Score: 0.48\tCurrent Score: 0.7900000140070915\n",
      "Noise Scaling: 0.19266091563311383, Memory size: 100000 and Num Steps: 302\n",
      "Episode 1947\tAverage Score: 0.47\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.19246825471748072, Memory size: 100000 and Num Steps: 128\n",
      "Episode 1948\tAverage Score: 0.47\tCurrent Score: 0.7900000121444464\n",
      "Noise Scaling: 0.19227578646276325, Memory size: 100000 and Num Steps: 316\n",
      "Episode 1949\tAverage Score: 0.47\tCurrent Score: 0.49000000953674316\n",
      "Noise Scaling: 0.19208351067630047, Memory size: 100000 and Num Steps: 182\n",
      "Episode 1950\tAverage Score: 0.46\tCurrent Score: 0.9000000134110451\n",
      "Noise Scaling: 0.19189142716562418, Memory size: 100000 and Num Steps: 372\n",
      "Episode 1951\tAverage Score: 0.46\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.19169953573845855, Memory size: 100000 and Num Steps: 14\n",
      "Episode 1952\tAverage Score: 0.49\tCurrent Score: 2.600000038743019\n",
      "Noise Scaling: 0.1915078362027201, Memory size: 100000 and Num Steps: 1000\n",
      "Episode 1953\tAverage Score: 0.49\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.19131632836651738, Memory size: 100000 and Num Steps: 188\n",
      "Episode 1954\tAverage Score: 0.50\tCurrent Score: 0.800000011920929\n",
      "Noise Scaling: 0.19112501203815085, Memory size: 100000 and Num Steps: 318\n",
      "Episode 1955\tAverage Score: 0.50\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.1909338870261127, Memory size: 100000 and Num Steps: 125\n",
      "Episode 1956\tAverage Score: 0.49\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.1907429531390866, Memory size: 100000 and Num Steps: 221\n",
      "Episode 1957\tAverage Score: 0.49\tCurrent Score: 0.800000011920929\n",
      "Noise Scaling: 0.1905522101859475, Memory size: 100000 and Num Steps: 313\n",
      "Episode 1958\tAverage Score: 0.50\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.19036165797576154, Memory size: 100000 and Num Steps: 123\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1959\tAverage Score: 0.49\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.1901712963177858, Memory size: 100000 and Num Steps: 147\n",
      "Episode 1960\tAverage Score: 0.50\tCurrent Score: 0.800000011920929\n",
      "Noise Scaling: 0.189981125021468, Memory size: 100000 and Num Steps: 344\n",
      "Episode 1961\tAverage Score: 0.50\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.18979114389644655, Memory size: 100000 and Num Steps: 69\n",
      "Episode 1962\tAverage Score: 0.50\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.1896013527525501, Memory size: 100000 and Num Steps: 67\n",
      "Episode 1963\tAverage Score: 0.49\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.18941175139979755, Memory size: 100000 and Num Steps: 52\n",
      "Episode 1964\tAverage Score: 0.49\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.18922233964839774, Memory size: 100000 and Num Steps: 14\n",
      "Episode 1965\tAverage Score: 0.49\tCurrent Score: 0.2900000046938658\n",
      "Noise Scaling: 0.18903311730874936, Memory size: 100000 and Num Steps: 107\n",
      "Episode 1966\tAverage Score: 0.49\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.1888440841914406, Memory size: 100000 and Num Steps: 44\n",
      "Episode 1967\tAverage Score: 0.49\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.18865524010724916, Memory size: 100000 and Num Steps: 69\n",
      "Episode 1968\tAverage Score: 0.49\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.1884665848671419, Memory size: 100000 and Num Steps: 67\n",
      "Episode 1969\tAverage Score: 0.49\tCurrent Score: 0.9000000134110451\n",
      "Noise Scaling: 0.18827811828227475, Memory size: 100000 and Num Steps: 357\n",
      "Episode 1970\tAverage Score: 0.49\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.1880898401639925, Memory size: 100000 and Num Steps: 52\n",
      "Episode 1971\tAverage Score: 0.48\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.1879017503238285, Memory size: 100000 and Num Steps: 14\n",
      "Episode 1972\tAverage Score: 0.48\tCurrent Score: 0.09000000357627869\n",
      "Noise Scaling: 0.18771384857350468, Memory size: 100000 and Num Steps: 31\n",
      "Episode 1973\tAverage Score: 0.48\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.18752613472493115, Memory size: 100000 and Num Steps: 70\n",
      "Episode 1974\tAverage Score: 0.48\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.18733860859020623, Memory size: 100000 and Num Steps: 31\n",
      "Episode 1975\tAverage Score: 0.48\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.18715126998161602, Memory size: 100000 and Num Steps: 105\n",
      "Episode 1976\tAverage Score: 0.48\tCurrent Score: 0.2900000046938658\n",
      "Noise Scaling: 0.1869641187116344, Memory size: 100000 and Num Steps: 106\n",
      "Episode 1977\tAverage Score: 0.48\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.18677715459292277, Memory size: 100000 and Num Steps: 146\n",
      "Episode 1978\tAverage Score: 0.48\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.18659037743832985, Memory size: 100000 and Num Steps: 91\n",
      "Episode 1979\tAverage Score: 0.48\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.18640378706089153, Memory size: 100000 and Num Steps: 186\n",
      "Episode 1980\tAverage Score: 0.49\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.18621738327383064, Memory size: 100000 and Num Steps: 144\n",
      "Episode 1981\tAverage Score: 0.49\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.1860311658905568, Memory size: 100000 and Num Steps: 239\n",
      "Episode 1982\tAverage Score: 0.49\tCurrent Score: 0.2900000046938658\n",
      "Noise Scaling: 0.18584513472466624, Memory size: 100000 and Num Steps: 107\n",
      "Episode 1983\tAverage Score: 0.51\tCurrent Score: 1.3000000193715096\n",
      "Noise Scaling: 0.18565928958994157, Memory size: 100000 and Num Steps: 526\n",
      "Episode 1984\tAverage Score: 0.51\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.18547363030035163, Memory size: 100000 and Num Steps: 69\n",
      "Episode 1985\tAverage Score: 0.49\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.18528815667005127, Memory size: 100000 and Num Steps: 191\n",
      "Episode 1986\tAverage Score: 0.50\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.18510286851338123, Memory size: 100000 and Num Steps: 128\n",
      "Episode 1987\tAverage Score: 0.49\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.18491776564486784, Memory size: 100000 and Num Steps: 56\n",
      "Episode 1988\tAverage Score: 0.49\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.18473284787922298, Memory size: 100000 and Num Steps: 239\n",
      "Episode 1989\tAverage Score: 0.49\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.18454811503134375, Memory size: 100000 and Num Steps: 219\n",
      "Episode 1990\tAverage Score: 0.49\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.18436356691631242, Memory size: 100000 and Num Steps: 311\n",
      "Episode 1991\tAverage Score: 0.49\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.1841792033493961, Memory size: 100000 and Num Steps: 30\n",
      "Episode 1992\tAverage Score: 0.49\tCurrent Score: 0.490000007674098\n",
      "Noise Scaling: 0.1839950241460467, Memory size: 100000 and Num Steps: 183\n",
      "Episode 1993\tAverage Score: 0.49\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.18381102912190067, Memory size: 100000 and Num Steps: 170\n",
      "Episode 1994\tAverage Score: 0.50\tCurrent Score: 0.800000011920929\n",
      "Noise Scaling: 0.18362721809277877, Memory size: 100000 and Num Steps: 305\n",
      "Episode 1995\tAverage Score: 0.50\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.18344359087468598, Memory size: 100000 and Num Steps: 164\n",
      "Episode 1996\tAverage Score: 0.50\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.1832601472838113, Memory size: 100000 and Num Steps: 33\n",
      "Episode 1997\tAverage Score: 0.50\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.18307688713652748, Memory size: 100000 and Num Steps: 145\n",
      "Episode 1998\tAverage Score: 0.50\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.18289381024939094, Memory size: 100000 and Num Steps: 129\n",
      "Episode 1999\tAverage Score: 0.50\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.18271091643914156, Memory size: 100000 and Num Steps: 292\n",
      "Episode 2000\tAverage Score: 0.49\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.18252820552270244, Memory size: 100000 and Num Steps: 69\n",
      "Saved Model: Episode 2000\tAverage Score: 0.49\n",
      "Episode 2001\tAverage Score: 0.49\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.1823456773171797, Memory size: 100000 and Num Steps: 15\n",
      "Episode 2002\tAverage Score: 0.49\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.18216333163986254, Memory size: 100000 and Num Steps: 239\n",
      "Episode 2003\tAverage Score: 0.48\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.18198116830822267, Memory size: 100000 and Num Steps: 90\n",
      "Episode 2004\tAverage Score: 0.48\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.18179918713991444, Memory size: 100000 and Num Steps: 200\n",
      "Episode 2005\tAverage Score: 0.48\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.18161738795277454, Memory size: 100000 and Num Steps: 14\n",
      "Episode 2006\tAverage Score: 0.49\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.18143577056482177, Memory size: 100000 and Num Steps: 203\n",
      "Episode 2007\tAverage Score: 0.50\tCurrent Score: 1.5000000223517418\n",
      "Noise Scaling: 0.18125433479425695, Memory size: 100000 and Num Steps: 587\n",
      "Episode 2008\tAverage Score: 0.50\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.1810730804594627, Memory size: 100000 and Num Steps: 166\n",
      "Episode 2009\tAverage Score: 0.50\tCurrent Score: 0.3900000061839819\n",
      "Noise Scaling: 0.1808920073790032, Memory size: 100000 and Num Steps: 144\n",
      "Episode 2010\tAverage Score: 0.49\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.1807111153716242, Memory size: 100000 and Num Steps: 30\n",
      "Episode 2011\tAverage Score: 0.49\tCurrent Score: 0.2900000046938658\n",
      "Noise Scaling: 0.1805304042562526, Memory size: 100000 and Num Steps: 116\n",
      "Episode 2012\tAverage Score: 0.50\tCurrent Score: 0.9900000151246786\n",
      "Noise Scaling: 0.18034987385199633, Memory size: 100000 and Num Steps: 373\n",
      "Episode 2013\tAverage Score: 0.50\tCurrent Score: 0.3900000061839819\n",
      "Noise Scaling: 0.18016952397814434, Memory size: 100000 and Num Steps: 155\n",
      "Episode 2014\tAverage Score: 0.50\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.1799893544541662, Memory size: 100000 and Num Steps: 63\n",
      "Episode 2015\tAverage Score: 0.50\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.17980936509971204, Memory size: 100000 and Num Steps: 101\n",
      "Episode 2016\tAverage Score: 0.50\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.1796295557346123, Memory size: 100000 and Num Steps: 83\n",
      "Episode 2017\tAverage Score: 0.50\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.1794499261788777, Memory size: 100000 and Num Steps: 32\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 2018\tAverage Score: 0.51\tCurrent Score: 1.8000000268220901\n",
      "Noise Scaling: 0.17927047625269885, Memory size: 100000 and Num Steps: 695\n",
      "Episode 2019\tAverage Score: 0.51\tCurrent Score: 1.1000000163912773\n",
      "Noise Scaling: 0.17909120577644613, Memory size: 100000 and Num Steps: 414\n",
      "Episode 2020\tAverage Score: 0.50\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.1789121145706697, Memory size: 100000 and Num Steps: 52\n",
      "Episode 2021\tAverage Score: 0.51\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.17873320245609903, Memory size: 100000 and Num Steps: 281\n",
      "Episode 2022\tAverage Score: 0.53\tCurrent Score: 2.600000038743019\n",
      "Noise Scaling: 0.1785544692536429, Memory size: 100000 and Num Steps: 1000\n",
      "Episode 2023\tAverage Score: 0.54\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.17837591478438927, Memory size: 100000 and Num Steps: 229\n",
      "Episode 2024\tAverage Score: 0.54\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.17819753886960488, Memory size: 100000 and Num Steps: 106\n",
      "Episode 2025\tAverage Score: 0.54\tCurrent Score: 1.1000000163912773\n",
      "Noise Scaling: 0.17801934133073527, Memory size: 100000 and Num Steps: 432\n",
      "Episode 2026\tAverage Score: 0.55\tCurrent Score: 2.600000038743019\n",
      "Noise Scaling: 0.17784132198940455, Memory size: 100000 and Num Steps: 1000\n",
      "Episode 2027\tAverage Score: 0.56\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.17766348066741514, Memory size: 100000 and Num Steps: 267\n",
      "Episode 2028\tAverage Score: 0.57\tCurrent Score: 0.9000000134110451\n",
      "Noise Scaling: 0.17748581718674772, Memory size: 100000 and Num Steps: 354\n",
      "Episode 2029\tAverage Score: 0.59\tCurrent Score: 2.500000037252903\n",
      "Noise Scaling: 0.17730833136956098, Memory size: 100000 and Num Steps: 960\n",
      "Episode 2030\tAverage Score: 0.61\tCurrent Score: 2.600000038743019\n",
      "Noise Scaling: 0.17713102303819142, Memory size: 100000 and Num Steps: 1000\n",
      "Episode 2031\tAverage Score: 0.60\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.1769538920151532, Memory size: 100000 and Num Steps: 4\n",
      "Episode 2032\tAverage Score: 0.60\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.17677693812313808, Memory size: 100000 and Num Steps: 272\n",
      "Episode 2033\tAverage Score: 0.60\tCurrent Score: 2.0000000298023224\n",
      "Noise Scaling: 0.17660016118501493, Memory size: 100000 and Num Steps: 756\n",
      "Episode 2034\tAverage Score: 0.61\tCurrent Score: 1.9000000283122063\n",
      "Noise Scaling: 0.17642356102382992, Memory size: 100000 and Num Steps: 713\n",
      "Episode 2035\tAverage Score: 0.59\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.1762471374628061, Memory size: 100000 and Num Steps: 79\n",
      "Episode 2036\tAverage Score: 0.59\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.1760708903253433, Memory size: 100000 and Num Steps: 54\n",
      "Episode 2037\tAverage Score: 0.61\tCurrent Score: 2.3000000342726707\n",
      "Noise Scaling: 0.17589481943501795, Memory size: 100000 and Num Steps: 909\n",
      "Episode 2038\tAverage Score: 0.63\tCurrent Score: 2.600000038743019\n",
      "Noise Scaling: 0.17571892461558292, Memory size: 100000 and Num Steps: 1000\n",
      "Episode 2039\tAverage Score: 0.64\tCurrent Score: 1.4000000208616257\n",
      "Noise Scaling: 0.17554320569096735, Memory size: 100000 and Num Steps: 540\n",
      "Episode 2040\tAverage Score: 0.65\tCurrent Score: 1.1000000163912773\n",
      "Noise Scaling: 0.17536766248527638, Memory size: 100000 and Num Steps: 408\n",
      "Episode 2041\tAverage Score: 0.64\tCurrent Score: 0.9000000134110451\n",
      "Noise Scaling: 0.17519229482279108, Memory size: 100000 and Num Steps: 337\n",
      "Episode 2042\tAverage Score: 0.64\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.1750171025279683, Memory size: 100000 and Num Steps: 257\n",
      "Episode 2043\tAverage Score: 0.65\tCurrent Score: 2.600000038743019\n",
      "Noise Scaling: 0.17484208542544033, Memory size: 100000 and Num Steps: 1000\n",
      "Episode 2044\tAverage Score: 0.65\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.1746672433400149, Memory size: 100000 and Num Steps: 101\n",
      "Episode 2045\tAverage Score: 0.65\tCurrent Score: 0.19000000320374966\n",
      "Noise Scaling: 0.17449257609667487, Memory size: 100000 and Num Steps: 68\n",
      "Episode 2046\tAverage Score: 0.66\tCurrent Score: 2.600000038743019\n",
      "Noise Scaling: 0.1743180835205782, Memory size: 100000 and Num Steps: 985\n",
      "Episode 2047\tAverage Score: 0.69\tCurrent Score: 2.600000038743019\n",
      "Noise Scaling: 0.17414376543705762, Memory size: 100000 and Num Steps: 1000\n",
      "Episode 2048\tAverage Score: 0.68\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.17396962167162056, Memory size: 100000 and Num Steps: 193\n",
      "Episode 2049\tAverage Score: 0.68\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.17379565204994896, Memory size: 100000 and Num Steps: 135\n",
      "Episode 2050\tAverage Score: 0.68\tCurrent Score: 1.0000000149011612\n",
      "Noise Scaling: 0.173621856397899, Memory size: 100000 and Num Steps: 395\n",
      "Episode 2051\tAverage Score: 0.69\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.1734482345415011, Memory size: 100000 and Num Steps: 88\n",
      "Episode 2052\tAverage Score: 0.68\tCurrent Score: 1.9000000283122063\n",
      "Noise Scaling: 0.17327478630695958, Memory size: 100000 and Num Steps: 741\n",
      "Episode 2053\tAverage Score: 0.68\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.17310151152065265, Memory size: 100000 and Num Steps: 90\n",
      "Episode 2054\tAverage Score: 0.67\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.17292841000913198, Memory size: 100000 and Num Steps: 312\n",
      "Episode 2055\tAverage Score: 0.67\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.17275548159912285, Memory size: 100000 and Num Steps: 109\n",
      "Episode 2056\tAverage Score: 0.67\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.17258272611752373, Memory size: 100000 and Num Steps: 204\n",
      "Episode 2057\tAverage Score: 0.69\tCurrent Score: 2.600000038743019\n",
      "Noise Scaling: 0.1724101433914062, Memory size: 100000 and Num Steps: 1000\n",
      "Episode 2058\tAverage Score: 0.69\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.1722377332480148, Memory size: 100000 and Num Steps: 156\n",
      "Episode 2059\tAverage Score: 0.72\tCurrent Score: 2.600000038743019\n",
      "Noise Scaling: 0.17206549551476677, Memory size: 100000 and Num Steps: 1000\n",
      "Episode 2060\tAverage Score: 0.74\tCurrent Score: 2.7000000402331352\n",
      "Noise Scaling: 0.171893430019252, Memory size: 100000 and Num Steps: 1000\n",
      "Episode 2061\tAverage Score: 0.74\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.17172153658923275, Memory size: 100000 and Num Steps: 279\n",
      "Episode 2062\tAverage Score: 0.74\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.17154981505264352, Memory size: 100000 and Num Steps: 243\n",
      "Episode 2063\tAverage Score: 0.77\tCurrent Score: 2.600000038743019\n",
      "Noise Scaling: 0.1713782652375909, Memory size: 100000 and Num Steps: 1000\n",
      "Episode 2064\tAverage Score: 0.77\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.17120688697235328, Memory size: 100000 and Num Steps: 4\n",
      "Episode 2065\tAverage Score: 0.77\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.17103568008538095, Memory size: 100000 and Num Steps: 18\n",
      "Episode 2066\tAverage Score: 0.77\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.17086464440529556, Memory size: 100000 and Num Steps: 68\n",
      "Episode 2067\tAverage Score: 0.77\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.17069377976089026, Memory size: 100000 and Num Steps: 280\n",
      "Episode 2068\tAverage Score: 0.80\tCurrent Score: 2.600000038743019\n",
      "Noise Scaling: 0.17052308598112936, Memory size: 100000 and Num Steps: 1000\n",
      "Episode 2069\tAverage Score: 0.81\tCurrent Score: 1.9000000283122063\n",
      "Noise Scaling: 0.17035256289514825, Memory size: 100000 and Num Steps: 715\n",
      "Episode 2070\tAverage Score: 0.81\tCurrent Score: 0.800000011920929\n",
      "Noise Scaling: 0.1701822103322531, Memory size: 100000 and Num Steps: 317\n",
      "Episode 2071\tAverage Score: 0.82\tCurrent Score: 1.0000000149011612\n",
      "Noise Scaling: 0.17001202812192084, Memory size: 100000 and Num Steps: 389\n",
      "Episode 2072\tAverage Score: 0.83\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.16984201609379893, Memory size: 100000 and Num Steps: 287\n",
      "Episode 2073\tAverage Score: 0.83\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.16967217407770513, Memory size: 100000 and Num Steps: 148\n",
      "Episode 2074\tAverage Score: 0.84\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.16950250190362742, Memory size: 100000 and Num Steps: 155\n",
      "Episode 2075\tAverage Score: 0.84\tCurrent Score: 0.490000007674098\n",
      "Noise Scaling: 0.1693329994017238, Memory size: 100000 and Num Steps: 182\n",
      "Episode 2076\tAverage Score: 0.84\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.16916366640232205, Memory size: 100000 and Num Steps: 112\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 2077\tAverage Score: 0.84\tCurrent Score: 0.7000000104308128\n",
      "Noise Scaling: 0.16899450273591973, Memory size: 100000 and Num Steps: 259\n",
      "Episode 2078\tAverage Score: 0.84\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.16882550823318382, Memory size: 100000 and Num Steps: 131\n",
      "Episode 2079\tAverage Score: 0.84\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.16865668272495063, Memory size: 100000 and Num Steps: 109\n",
      "Episode 2080\tAverage Score: 0.84\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.16848802604222568, Memory size: 100000 and Num Steps: 15\n",
      "Episode 2081\tAverage Score: 0.85\tCurrent Score: 2.0000000298023224\n",
      "Noise Scaling: 0.16831953801618346, Memory size: 100000 and Num Steps: 780\n",
      "Episode 2082\tAverage Score: 0.85\tCurrent Score: 0.490000007674098\n",
      "Noise Scaling: 0.16815121847816727, Memory size: 100000 and Num Steps: 181\n",
      "Episode 2083\tAverage Score: 0.84\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.16798306725968912, Memory size: 100000 and Num Steps: 71\n",
      "Episode 2084\tAverage Score: 0.87\tCurrent Score: 2.500000037252903\n",
      "Noise Scaling: 0.16781508419242944, Memory size: 100000 and Num Steps: 968\n",
      "Episode 2085\tAverage Score: 0.87\tCurrent Score: 1.2000000178813934\n",
      "Noise Scaling: 0.167647269108237, Memory size: 100000 and Num Steps: 453\n",
      "Episode 2086\tAverage Score: 0.88\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.16747962183912876, Memory size: 100000 and Num Steps: 242\n",
      "Episode 2087\tAverage Score: 0.88\tCurrent Score: 0.10000000149011612\n",
      "Noise Scaling: 0.16731214221728963, Memory size: 100000 and Num Steps: 45\n",
      "Episode 2088\tAverage Score: 0.90\tCurrent Score: 2.600000038743019\n",
      "Noise Scaling: 0.16714483007507233, Memory size: 100000 and Num Steps: 1000\n",
      "Episode 2089\tAverage Score: 0.90\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.16697768524499726, Memory size: 100000 and Num Steps: 199\n",
      "Episode 2090\tAverage Score: 0.91\tCurrent Score: 2.0000000298023224\n",
      "Noise Scaling: 0.16681070755975227, Memory size: 100000 and Num Steps: 777\n",
      "Episode 2091\tAverage Score: 0.92\tCurrent Score: 1.3000000193715096\n",
      "Noise Scaling: 0.16664389685219252, Memory size: 100000 and Num Steps: 509\n",
      "Episode 2092\tAverage Score: 0.93\tCurrent Score: 1.3000000193715096\n",
      "Noise Scaling: 0.16647725295534033, Memory size: 100000 and Num Steps: 511\n",
      "Episode 2093\tAverage Score: 0.92\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.16631077570238498, Memory size: 100000 and Num Steps: 14\n",
      "Episode 2094\tAverage Score: 0.93\tCurrent Score: 1.4000000208616257\n",
      "Noise Scaling: 0.1661444649266826, Memory size: 100000 and Num Steps: 545\n",
      "Episode 2095\tAverage Score: 0.93\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.16597832046175592, Memory size: 100000 and Num Steps: 223\n",
      "Episode 2096\tAverage Score: 0.93\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.16581234214129417, Memory size: 100000 and Num Steps: 108\n",
      "Episode 2097\tAverage Score: 0.93\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.16564652979915287, Memory size: 100000 and Num Steps: 14\n",
      "Episode 2098\tAverage Score: 0.93\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.1654808832693537, Memory size: 100000 and Num Steps: 15\n",
      "Episode 2099\tAverage Score: 0.92\tCurrent Score: 0.0\n",
      "Noise Scaling: 0.16531540238608436, Memory size: 100000 and Num Steps: 15\n",
      "Episode 2100\tAverage Score: 0.93\tCurrent Score: 1.600000023841858\n",
      "Noise Scaling: 0.16515008698369826, Memory size: 100000 and Num Steps: 607\n",
      "Episode 2101\tAverage Score: 0.94\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.16498493689671456, Memory size: 100000 and Num Steps: 242\n",
      "Episode 2102\tAverage Score: 0.94\tCurrent Score: 0.4000000059604645\n",
      "Noise Scaling: 0.16481995195981786, Memory size: 100000 and Num Steps: 144\n",
      "Episode 2103\tAverage Score: 0.94\tCurrent Score: 0.5000000074505806\n",
      "Noise Scaling: 0.16465513200785803, Memory size: 100000 and Num Steps: 185\n",
      "Episode 2104\tAverage Score: 0.95\tCurrent Score: 1.1000000163912773\n",
      "Noise Scaling: 0.1644904768758502, Memory size: 100000 and Num Steps: 454\n",
      "Episode 2105\tAverage Score: 0.95\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.16432598639897433, Memory size: 100000 and Num Steps: 128\n",
      "Episode 2106\tAverage Score: 0.95\tCurrent Score: 0.30000000447034836\n",
      "Noise Scaling: 0.16416166041257535, Memory size: 100000 and Num Steps: 124\n",
      "Episode 2107\tAverage Score: 0.95\tCurrent Score: 1.4000000208616257\n",
      "Noise Scaling: 0.1639974987521628, Memory size: 100000 and Num Steps: 548\n",
      "Episode 2108\tAverage Score: 0.95\tCurrent Score: 0.6000000089406967\n",
      "Noise Scaling: 0.16383350125341062, Memory size: 100000 and Num Steps: 243\n",
      "Episode 2109\tAverage Score: 0.97\tCurrent Score: 2.600000038743019\n",
      "Noise Scaling: 0.16366966775215722, Memory size: 100000 and Num Steps: 1000\n",
      "Episode 2110\tAverage Score: 0.97\tCurrent Score: 0.09000000171363354\n",
      "Noise Scaling: 0.16350599808440505, Memory size: 100000 and Num Steps: 36\n",
      "Episode 2111\tAverage Score: 0.97\tCurrent Score: 0.20000000298023224\n",
      "Noise Scaling: 0.16334249208632065, Memory size: 100000 and Num Steps: 73\n",
      "Episode 2112\tAverage Score: 0.97\tCurrent Score: 1.2000000178813934\n",
      "Noise Scaling: 0.16317914959423432, Memory size: 100000 and Num Steps: 456\n",
      "Episode 2113\tAverage Score: 0.99\tCurrent Score: 2.600000038743019\n",
      "Noise Scaling: 0.16301597044464008, Memory size: 100000 and Num Steps: 1000\n",
      "Episode 2114\tAverage Score: 1.00\tCurrent Score: 1.2000000178813934\n",
      "Noise Scaling: 0.16285295447419545, Memory size: 100000 and Num Steps: 464\n",
      "Saved Model: Episode 2114\tAverage Score: 1.00\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xd4W+X58PHvLXklzo6dHWeHMBJGAoQdKFBWgZZS6KCFDl5KC3TRH4UW6KClpYVC2RQoUDYUCCVAQxoI0JAFScgge9kZzrIdb437/UPDki1Lsq0jyfb9ua5cls55dM6jE/u5zzOPqCrGGGMMgCvTGTDGGJM9LCgYY4wJs6BgjDEmzIKCMcaYMAsKxhhjwiwoGGOMCbOgYIwxJsyCgjHGmDALCsYYY8JyMp2BtioqKtLRo0dnOhvGGNOpLFmyZI+qFidK1+mCwujRo1m8eHGms2GMMZ2KiGxJJp01HxljjAmzoGCMMSbMgoIxxpgwCwrGGGPCLCgYY4wJs6BgjDEmzIKCMcaYMAsKxhiTBZaXVrC8tCLT2eh8k9eMMaYrOv/eDwHYfPu5Gc2H1RSMMcaEWVAwxhgTZkHBGGNMmAUFY4wxYRYUjDHGhFlQMMaYDqj3+Kj3+ACoqG1M6jPlVfXsqKyjqt7D1r21VNZ6wvuqG7zh49Y1+sKvd1bWo6opzn1LNiTVGGM64JCb30JEuPMrh3Pdc0uZ+cMTmDKiX6vpF23ex8UPzm91/2G3vM3m28/l6Nve4UC9l823n8ukX70FwP+dNYnvzxiX8u8QyWoKxhjTAX4Fn1/5cP0eAFbvqIqbftm25CaoHaj3ttj2zupdbc9gG1lQMMYYE2ZBwRhjTJhjQUFERorIXBFZLSIrReS6GGlmiEiliCwN/rvZqfwYY0w6pKEv2FFOdjR7gZ+q6sci0htYIiKzVXVVs3Tvq+p5DubDGGNMkhyrKajqDlX9OPj6ALAaGO7U+YwxJhuIZDoHHZOWPgURGQ0cCSyIsfs4EVkmIm+KyKHpyI8xpvv6zeuruOPtz1i76wCn/eXdpOcWpNPmPTUZO7fj8xREpBfwMvAjVW0+VutjYJSqVovIOcCrwIQYx7gSuBKgpKTE4RwbY7qyxz7cBMDWfXVs3F3De2t3c8ERqWvESEWfwqMfbOr4QdrJ0ZqCiOQSCAhPq+q/mu9X1SpVrQ6+ngXkikhRjHQPq+o0VZ1WXFzsZJaNMaZdhE7ebhTk5OgjAR4FVqvqna2kGRJMh4gcE8zPXqfyZIwxIaEiPBtHC2WyX8LJ5qMTgMuAT0VkaXDbjUAJgKo+CHwZ+L6IeIE64FJNx+IexhiTYkrXKLocCwqq+gHEr0+p6r3AvU7lwRhj0s1GHxljTCcUKrxTfYfvZFtHOuKNBQVjTLeU6j6FVHY0t3akdDRQWVAwxhgTZkHBGGNSIB0dzdZ8ZIwxDgmOhs/KIamZZEHBGNMthfsUUna8FPYpZHAIkwUFY4wxYRYUjDHGhFlQMMZ0T6F5CtapEMWCgjGmW+oqC9ilmgUFY4zJMplcKsOCgjGmW7PGo2gWFIwx3ZKkekxqCg+XyaYtCwrGmG4p1cVuZ18dNcSCgjHGpEBXGcRkQcEY062les2izl5hsKBgjOmWnGruSUmfgo0+MsaYzEjZ8xTSUJCn4xwWFIwx3ZJNXovNgoIxpltLVY+CdTQbY0wnls1DSDOZNQsKxphurTP1KaSDBQVjTLfUVQrxVLOgYIzp1tLxbOXOxIKCMaabyt6qgs1TMMaYDMnGUUP2jGZjjEkz61OIzYKCMcaYMMeCgoiMFJG5IrJaRFaKyHUx0oiI3CMi60VkuYgc5VR+jDEmlixsPcqoHAeP7QV+qqofi0hvYImIzFbVVRFpzgYmBP8dCzwQ/GmMMY4Ktx5lYadCl5y8pqo7VPXj4OsDwGpgeLNkFwBPasBHQD8RGepUnowxJiSb+xSq6j0ZO3da+hREZDRwJLCg2a7hwLaI96W0DByIyJUislhEFu/evdupbBpjTFZ4dmFTsbi3uiH8Oh2L+DkeFESkF/Ay8CNVrWq+O8ZHWtTlVPVhVZ2mqtOKi4udyKYxppvKvsajaJV1TbWGdEy0czQoiEgugYDwtKr+K0aSUmBkxPsRwHYn82SMMdB0153qLoUs7KJoEydHHwnwKLBaVe9sJdlM4JvBUUjTgUpV3eFUnowxxilpechOGpqPnBx9dAJwGfCpiCwNbrsRKAFQ1QeBWcA5wHqgFrjCwfwYY4xjnKohpLvi4VhQUNUPSDCySlUV+IFTeTDGmNaE7uw1xaV5No9qSobNaDbGdEuhsjvVd+JO9il0+o5mY4zpyjw+f/j1c4u2xUnZfunuuLagYIwx7VRVl45JZk1RoUvMUzDGmGwUWp66sw8hTTULCsaYbq0jMSGTzz1wigUFY4zJYtanYIwxJmMsKBhjuiWn5il0dhYUjDHd0tMLtnb4GO3pUZizurxN6dMdsiwoGGO6pUavP3EiB8zfuDcj502WBQVjjDFhFhSMMSaL2egjY4zpxNKxPpGTLCgYY0w7pWPuWrqDjAUFY4wxYRYUjDEmSeUH6tlX0wjApj01NPpajmBKx6J1TnLyyWvGGNOlHHPbHACW3Xwmp/75XU4/eLDj54zqaE5DvLGagjHGtFFNoxeA+Rv2tNhnHc3GGGO6DAsKxhjTRuF1k9JwLpunYIwxnUSsJv5UdzTbkFRjjOkkYhXX1qdgjDHdTKg20BVX3bagYIxps9PvfI9fv74y09lImyN/8x/um7s+/H76H+ak7dzn3vNB2s4FFhSMMe2wvryaxz/cnOlspM3+Wg93vL2mxfbO3lQUiwUFY4xpJ2s+MsYY06U5FhRE5DERKReRFa3snyEilSKyNPjvZqfyYowxTuiCFQVH1z76B3Av8GScNO+r6nkO5sEYY+LqUBNQF4wKjtUUVHUesM+p4xtjjEm9pIOCiJwoIlcEXxeLyJgUnP84EVkmIm+KyKEpOJ4xppt7bWkZry0t454569AkqgGRD8op3V/LfXPXJ/U5aNvoo/ID9fz1nbVJp48lHYtyJ9V8JCK3ANOAg4DHgVzgn8AJHTj3x8AoVa0WkXOAV4EJrZz/SuBKgJKSkg6c0hjT1V333NLw69MmDeKw4X3jpo8s/7/7xGI+23mAL0wZRsnAninN109fWMb761quqtoW6WitSram8EXgfKAGQFW3A707cmJVrVLV6uDrWUCuiBS1kvZhVZ2mqtOKi4s7clpjTDfS1v6C0JLYydYA2nL8eo+vbZnJkGSDQqMG6lMKICKFHT2xiAwRCVTcROSYYF72dvS4xhiTLjHXPnLwdj5rmo+AF0TkIaCfiHwP+DbwSLwPiMizwAygSERKgVsINDuhqg8CXwa+LyJeoA64VJNtyDPGmBSRzv30zJRLKiio6p9F5AygikC/ws2qOjvBZ76aYP+9BIasGmNMxsS6FU12+euueB+bMCiIiBt4W1VPB+IGAmOM6e46e80jYZ+CqvqAWhGJ34VvjDFdxMLN+/D6/AnTpbtPIR2S7VOoBz4VkdkERyABqOq1juTKGGMy6GcvLmN9eTU3nD0pbrrOHgBiSTYovBH8Z4wxXVZkIf9pWYVjx85myXY0PyEiecDE4KY1qupxLlvGGJN+naXgdlKyM5pnAE8AmwkMlR0pIt8Krm9kjDGmi0i2+egvwJmqugZARCYCzwJTncqYMcakW+QQ0+5aa0h2RnNuKCAAqOpaghPRjDGmq+imcSBKsjWFxSLyKPBU8P3XgSXOZMkYYzIjsnZgNYX4vg+sBK4FrgNWAVc5lSljjEmX22at5ssP/A9o21LYyZhxx1yuefaTlB7TackGhRzgblX9kqp+EbgHcDuXLWOMSZ/FW/YDqa8dbN5by+vLtqf2oA5LNijMAXpEvO8BvJP67BhjTOZke4tROpbQSDYoFISefQAQfJ3aJ1AYY0yGRfUpZGGISEc/R7JBoUZEjgq9EZFpBJa7NsaYLsSGpCY7+uhHwIsisp3AVRsGXOJYrowxJgNSEQhaO0QqYkzGm49E5GgRGaKqi4BJwPOAF3gL2OR89owxJn26aeUgSqLmo4eAxuDr44AbgfuA/cDDDubLGGPSav6GvSl5aE4nf5xCwqDgVtV9wdeXAA+r6suq+itgvLNZM8aY9PnqIx9F1RSysdZQ3LvA8XMkDAoiEup3+Bzw34h9yfZHGGNMpxCrotDW2oOTweSKE0Y7ePSARAX7s8B7IrKHwGij9wFEZDxQ6XDejDEmrbL9mcvpaJqKGxRU9TYRmQMMBf6jTVfMBVzjdOaMMSadtNU33UfCJiBV/SjGtrXOZMcYYzIoZvNR2w7h5N28pGFMarKT14wxpsuL7mhuX1Whs1cwLCgYY0xQrD6FbCrk09GnYEHBGGOCsikAZIoFBWOMCbKH7FhQMMaYuLJpmGrG1z4yxpjuxJ9FASBTHAsKIvKYiJSLyIpW9ouI3CMi60VkeeTS3MYYkwmxQkI2hQlJQ1ezkzWFfwBnxdl/NjAh+O9K4AEH82KMMYlpzJfdimNBQVXnAfviJLkAeFIDPgL6ichQp/JjjDGJZHPz0TGymtxq55/3nMk+heHAtoj3pcFtxhgT14+fX8r4G2cllXbznhqqG7xJpfX6I5+8psGfbc9fqgl+nsr7AwNXPe74uTIZFGI1jsW8/CJypYgsFpHFu3fvdjhbxphs98onZVEFeDwz/vwulz483+EcJdaRUUxFVJIvXjy9R6YwR7FlMiiUApHfcAQQs26kqg+r6jRVnVZcXJyWzBljuo4VZVWZzkKHfDvnLQAaeznfmJLJoDAT+GZwFNJ0oFJVd2QwP8YYE6bhn5lvP7rQ/SEAtUOOdfxcjj0oR0SeBWYARSJSCtwC5AKo6oPALOAcYD1QC1zhVF6MMSZtUtoJofw+51GGyj5m+6YyLL9XCo8dm2NBQVW/mmC/Aj9w6vzGGJMKmepoLqCBzwqa7pV/7b2Mh9JwXpvRbIwxMbQ7GKRoLYojXevDrx/wfoFSHZSS4yZiz1k2xmSV/6zcyb6aRi49piS87aUlpRTmuTl7csemMqVrHaOn5m9m5ICeHTrGya7lAPzScwUv+U4G0jOj2YKCMSarXPnUEoCooPCzF5cBsPn2czt07NL9dUmnbXf4UOVXr61s76fDjpANAPzTdzrpeZJCgDUfGWNMlsmnkePcq5jrO5zIgGCrpBpjTIZloqP5GNdnAMzxp3+dUAsKxhiTSh28ne9BPU/l3Q7AIv9BqchRm1hQMMaYWNpbRehg1eLu3PvCr7fo4Kh91nxkjDEZkokZzQOp5HOuj/nIfzDH1t9LPflpO3eIBQVjTErUNfqYv2Fvh44Ra8hoRW1ju4/30cam/GyvqOOznQfafaxk/fk/a1tse3dNeVKhZYZrGW5Rfuu5jF0MaLHfhqQaYzqNm175lH99UsZ7189g1MDCdh3jmYVbo97Xe3wc8ZvZ7c7TbbNWh18ff/t/23WMtrYGVdZ5Wmy7/PFFSX12gquMBs1hlZYkTuwQqykYY1IidBd+oD65ZxfEsm1f9DyCpxdsbSVlV6Qc5VrLTh2AtlI0W5+CMaZbad5+7/X546d3cLxouoeinuBawTGuNdRSkN4TN2NBwRiTPTTu25bJ01Bwpys2lEg5AL/1fiNNZ4zNgoIxJiWcaNpIVOhn/kkHqXOwbKVBc1jin9hqmnQsdmFBwRiTNdpayPudbD5Kc8g53LWBBf6DaSAvredtzoKCMSZrtLWPwOdXGr3x+x06Kl0rqxZLBTu15TDUSOnoaLYhqcZ0c+UH6jnmtjnc8eUpXDyt4w+GD5WhK8oqOe9vH4S3Txzci//8+BQA5n5WzhX/WMRhw/uwoqwqvPpp8/L3j299FvX+g3V7WLh5X/j9lx/8HzUNPub+bAY7Kus47g/tG3Yay4qyKl5eUsqZhw5OnLgDcvDSk3qGyT5209fRcyXDagrGdHMbd9cA8OKS0g4dp/ld7Ifr90S9X7urOvx61qeBx7GvKKtq0zlmr9rJI/M2ht+vKKti055A/tc4MDHtX5+UOt6IdFPO0ywvuBKATdqx50WkggUFY0xKtKWVJccdux2kIwWwE4W32+V8ERl6mA7Ax/4JCVI7335kQcEYk3ZuVytBIUHJLiKttqs70faf00o+U+Vg2cI41w42+QdzXsPv2KjDHD1fMiwoGGNSoi2doDkO3IE70R/sdomjcyHezP8FALniY4WOTZjeZjQbYzqdZIZytlpT6EAjkBOFt9M1hZDLGn+RlvMkw0YfGWPSrrXCNrJgb2tzkDN9CuLQDDnlmdzbAHjUe3bSHcw2ec2YLsrj8/Pzl5axdW9twrTlVfX85Pml1Ht8LfZV1nq47rlPOFDfcmXObHT/u+u5b+56Pmg2MimWN4IjlCLtqKyjtrHldVheWsFtb6xKSR4j/Xv5DjbuqU6csI1GyB6OdwfyW6U9U378jrCgYEwGLNmynxcWl/Kzl5YlTPu7N1bzr0/KeGvFzhb7Hpy3gdeWbufJ+Vtifrbe42P0DW/wxP82dzTLCSWz1v+f3lrDHW+vYeX2xENRf/jMJy22vb1yV8y0Fz84n81JBNj2+OL9/0v5MfvTNHx2N/1SfvyOsKBgTJYLtV7E6mQMbWqtqaWiNlCDuP/d9anPmAPaO4Kos62BVCSVAMz2HcUrvhOS/pykoafZgoIxWS5eQRkqI9K9zLNT2v01Otn3Hy2BWt9Nnu9Ql+GlspuzoGBMlotX3iVqskn3om6Z4uTCeE44yfUpZTqQcvq36XOdvqNZRM4SkTUisl5Eboix/3IR2S0iS4P/vutkfozplILlXbymg0RFYjqe7RvSkfK5vZ/tTCFhMPs4zb2URf6DMp2VmBwbkioibuA+4AygFFgkIjNVtfkQgedV9YdO5cOYriJWsZ6NzUcdW6qinX0K2XQBEjjPPR+A133HZTgnsTk5T+EYYL2qbgQQkeeAC4DUjxszJsuU7q9lb3Ujh4/s+MiSUEEZq6KwJTji5t215Vx3est1cxKVlR9t3Mu+msZ25215aQULN+3juyc1zcZduGkvK7dXtql437C7Go/Pz4KN+xInjsGfJTFhomzjXPcCnvOeyg4G8rucR6mlgAotZJBUcJ77I4okMPJqsw5p8/E7+9LZw4FtEe9LgWNjpLtIRE4G1gI/VtVtzROIyJXAlQAlJSUOZNWY1Drxj3MBwktCd0SoYI/VBDRz2XYAPtlawcbd1Ywt7hXzGLEKE1Xl0oc/6lDezr/3QwDOmzIsfI7fzwosd10yIPnx95/7y3sdykemufExWnbyn/z/A+C6nH/xmu94LnC3Ppy1PUEhHZzsU4gV05rH89eB0ao6BXgHeCLWgVT1YVWdpqrTiouLU5xNY7JbOCgkuEusbvC26bipvLuONbFuZ1V96k6QZuOkjHwCNSgXftz4OMu1kH7EXp775znPMSf/+qhtrQWE570zmFL/CD7cbc5XOvqGnKwplAKRT+wYAWyPTKCqeyPePgL80cH8GNMphZuP2vXZ1vnaGRVitd+no1nDaXl4OMG1gsfz7oibbnT9My22fdXd9HCfo+vvp548LnB/yAYdxnL/WGrJZ6zsIB8Pq3R0qrOeUk4GhUXABBEZA5QBlwJfi0wgIkNVNTSX/XxgtYP5MaZTSramEE+sj7Y3KMTi6pRRQbk95xHGuHZyrOuzxMmD3s37Md/w3EipBlotPu9aRB+pA2Ceb3Lw6WnCP31nRH1ugw5PWc6d5FhQUFWviPwQeBtwA4+p6koR+Q2wWFVnAteKyPmAF9gHXO5UfozprJqK7tQWvL72zh5u5WOdLSwcLFu5NOfdmPtG1z/NCNnDWa6FvO2fxsO5d7HAP4nLc/7DaNcuznIt5O++QH/RD3JeBeC6xqt5zX+io3nu7B3NqOosYFazbTdHvP4FkD1rxhqThZKtKbS1jE9lTaEzVhRGyO7w69X+Ei5r/AU9pJ5y7Q8IpVocLvjPbrydPtRQSD0X58zjl7lP875/Mmu0hL7U8KrveMcDQrrY0tnGxKGq+PxKjjv2mAyfX3FJ4KfbJe1emyb+eQKFt9/f9FNp2bavgNfnjzpG8zR+v4Zn/zYPCn6/4vcrrohlrb0+f4vv5fH7W+RQRPA2DzJZMky0NTfkPAvAtPoH2EPfwEbt22r6Kgq53nsVF+fMA+DKnDd43zeZUa5yZnunOp7fdLFlLoyJ46/vrGP8TW9SF2O5ZoBxN87ie08uZvxNb/L39ze1+zw3vbqC8Te9GXNfqFz//tMfs3bXAU7581zG3TirRfpnF2xl/E1vsiti1E9TLUPYfaCBk++Yy/ib3mT8TW+2WG578Zb9nPe3D8Lvq+o9jL/pTe5/d0N42/ryag765Vst8njVU0tarHza6GsZPLKD8pOcFxjnCnRn7qV3mz59TP19AFzkfp+/5t0PkPTzEDoDCwrGxPHMwq0AcZ9X8M7qcgBeWlLa/vMsCJwn1sieyC3LSyvZtq8u5jFeWVoGNE1oa27GHXMp3d/02ViT1lbtaCrYdx9oAKK/14qyypjH/rSV7dlmjOzg5bxbuTbYDzC1/gG0jcVg8/WKVvhH87TvcynLYzydvk/BmM4uVEYn0xKSij/YQBNS9IEiA0Xc5Ry0ZZrI5DXNajuJ+hRCx4nMTSr7IdKhmP3ckfsw//CdSR9quSfvvvC+axt/wF5aby6Kp0p70EfqmNHwFzZ3oVoCWFAwxlGq2qZ+Bq9fyWk2pynZYjjWukHxVg9t0QfQ4rOBn5HZb++IpXS70PVBuGkHYIY7+mFGq/0jmelP/jkGzU1peLTdn812FhSMiSNUILa3EuBXcLfhwx6fn4Lc6KgQWQ7HCzCB2a7RQSheEe5PWFMI/Iycg5CtC8/l4aGRXADuyf0b5wcXnQOo1J7UUMAQ9vN1z43Uaj7bdFCmstoh6XjIjgUFYxwUGpWULK8vfp9CvEI5Vk0hXvpEd/2h40UGhWzrOxb8jJJdzMq7kZ7SELVvnm8yP/BcxwGy6xnI2c6CgjEOSvbhLy4J1CpiDffsyN15vMpAov6BUFYib06z7WE2v8r5J9/OiR4N5VE3V3l+xBx/1xkmGpKO6SAWFEyX8ff3N5LrdvGt40en5Xx7q6PvTD/beYDrX1zGHRcfHt7m8yvbK+q48ZVPufdrR1Hb4OXbTyxiRVlglM/aXQe45KH54cLb61N+/fpKVm2v4twpQ7noqBG8v25P+HjXv7S81fx4grWMrzw0n++eOIbRRYVMHzsAgLKKliOWLn98UczjvLumnBkHDQoHgNpGH5NveZupo/tz2qTsaHZx4eeH7lfDAeGLDb/mEx2PoLjQdi02ZwIsKJgu43dvBJbOciIoRN4fL9myj4semB8z3YtLSqMKTp8qd85ey7trdjPr0x2s2l4VDggAFbUeFmxqeoaAz688/uFmABZs2kdhXvv+RP/+QfvnTFz++KKoJb+37gs+s2HNbt5ds7u1j6XVFe43+UnuSwBMr/8bOxkIgCLEnlFikmVBwZgkRDabhOYltOb5xU2PBGnemdvWfsK29EekmhNNRUPYyx764k1Q9Ljw4282f+AK95vckvtU1LZzG34fDghdRc88N7WtTJa0eQrGZInI8jFRWRnZVt+83T7RevjNC+JMrimUyikJfajh33k3UuLazYe+Q/m656ZW0x4tn/Fi/m/Y7B/MI75z6Uc1gvKz3Bej0l3XeDUrs3wZ6vbI9DJSFhSMSUJb7pojRxA1H+GTqJBvHkS6Qk0hn0Y+zv9/5Eig5/oE90qmetcw1bWW5TqOo+UzSrWYd/xTOdu9gD/lPgLAaNcubnM9FnWscxp+zzDZSx9quswCdM0lHnbsLAsKxiQheq5A/LSRBXuMwURxeZoNSc3kcwoSzWOIJw8PLvx81f3fcJPPU97Tect/NE/n/YGX838d9/M/abyK5TqWm3Ke5tTgxLOrG69llY7O+ofUdJTVFIzpBNrSfOSNiAQtagoJztPojY4imQwKbV3SwoWfi93vcXPOkxQ2mzPwD++Z3Oq9HICfe77H4bKRpTqOkVLObu3HNNdajpR1+HFxi/dy3vMHRnBd4fk/aH3Zqa4pzn+59Sl0UVv31lJ+oJ5powdkOispkez3WV9+gGXbKhlbXMiRJf3jpu2IT7bu54fPfMIVJ4zmjEMGs6y0ktoGL5ceUxJOs3RbBb3ycxg/qOlB95v31LCnuoGyijrOmTyU99ftDi8K9/ry7UwfO4ABhfm8vy7+CJyPt1aEX7+zahezV+0CYMnm/eyvbbkIXaTZq3ZGvb/qn0uS+9Ipduzv32FXVUPihEGF1HFLzpN8Jee98Lb3fYexWkdxl/ci6igIb3/BdyovcGrU55/yndnxTHcRma4pSLZOW2/NtGnTdPHixZnORoeMvuENgKhhf51Zst8nlA7gxauO4+gYQaTB6+PmV1fSr2cu08cN5NSDWh8XP3vVLsr213L5CWPYWVnP9D/MiXv+9bedzY9fWMaCjXspDxb2Rb3y2VPdwMTBvVi7qzqc9jsnjuHRDgzrzDRXcKbvdh1IA3mOnEPw8xX3e/wx2AcA8K7vcL7tub7FyCGTvD4FOVTVe2PuW3jj5xjUpyDmvkREZImqTkuUzmoKJiMufnB+zCDy1oqd4SGdD83bGDfQfO/JwM3B5SeM4ebXViQ858LN+3h92faobXuCE9AiAwLA9hiTvbKXcpSsY6SUc5hrMye7ljNWdpArgWGNu7Uv6/zD2U0/dmp/KrUQEP7nP5SlOr5dZzzPNZ978/4Wfr9Xe3OX98v803c6mb/XTc65U4byxvIdiRMmYePvz+Ho295hb4zlyBPJc7v4+7emUdQrn3PueT/u4ILWHvaUShYUTJeQ1EiZzlUpTiifRn6T8w8uafac4b3am9f8J7BH+3CF+y2KpZJidyWlWsQI2dPiOJv8g+kl9bzsO4kqLWSTDsGPkIuPef7JVNELwc9Y2UGNFnCIa0s4IDzg/QKPe89q8YyBdFh2y5kc/uv/0Ds/hwMNse+s4/neSWNTFhRcHRgldvWp4zh5YjGrgg8pihcU0jEazYKCMTFk2xo/fajhAveHnOJaRh5e+khJMUOSAAAWcklEQVQNR7g2hve/6Tuamb7jWasj2KDDw9sf9p5HAY1spwiAKbKBAXKAkVLO9Tkv0EdqqSePMbKLq3L+HfPcZTqQHjQwQKJrUw95z+WP3q868G2T09FO11T/H7c3Py3mssQ5UG5bltxtJwsKpttoy7MAOroaaCF1HOFaz0L/wXhwE6tJxYU/8KzlYPt7Po0oghd3RJu88h33LH6a81LUKqD7tRf7tRe/936Nl30nt9qGv48+Ue+X67hwjekp3xnBrRI+fx9qGO/azhTZiBc3B8k2csVLo+ZSLBXUUkADOSzyT+Jl38ntvTxZIVv6U1sOQ249bY7Lmo9MN+PkevGxlqVuTaK7yDw8eHHzNfccxksZfaWGFf7RnO76hMNdG6IKcJ8KDeSxTodTRz55eMjBxxRX/I5sr7rCE742+ofw08bvs1zHko+HWtrX2Rgt+lo3kMdu8tjt7898Dk3B8bNbtjxEzhu8Awn96scbhpxjzUfOqGv04fX76V2Qi9fnx+vXqAebNHh91Hv89CnICRdSfr/S4PXTI89NRW0jdR4ffQpyKcxvuoRV9R7y3C58weNVh0YQCFQ3eHGLRFUx6z0+Kmo99CrIoVd+DqpKncdHz3YugpYsn19pDH4XCDx/2O0S8twuquq95OW48Hj9eHx++vXMw6+B76OqVNR6yMtxhZ/a1eBtWqMlNHyzrtFH3565uF3CgXoPFbUeehe0/E4ryirplZ9Djltwu4R6j591uw5Epdm0p4Zct4Tvphq9fgb3yY8az7++/ADry6ObNmL5eOt+AIayl6Gyl2KpJJ9GxrvKcKGMlR1Mkq0s8U9k2Y7j+YZ7F32oYbprNQXSiB8XxVQgKGNdO1sc/4vuDynXfjzvm4ELP7n4KNMiCqSRQuo5xLWFvlQzRnZSIB5W+kexQYfhIYciKqmmgDX+Ek5yL6dRc/HixoWfOf6j+Ifv84QK8VpbARRo26NSY+nI5LxUav4EvHhBoSN9F8nqdkFhb3UDU3/3DgBvXHsid81eyzury3nl6uPDY+cP+mVgOd4bzp7EVaeMA+CWmSt56qMt/PuaEznvbx+EjxcaHbNtXy0n/Wlum/Iy6VdN68D/+5oTmbO6nLveWcvMH55Afo6bg4b0bv8XjeNXr63gmQVb2fSHc/hs5wHOvvt9oGl4Ziybbz+X655bysxmo3ciHX3bO23KR+R1bM2pf343YZrT75zXYpsLP2NlO7n4cOPjRNcKhs/bycL8pQySihhHgQotZLsWcXHOPC5umEfwQV4c0B6s0+F4ELYGn9i1xDuRcvqx3j+cD/yTGS572Kn92UtfPB38s7rH96UOfb67yAuOxDl6dH/mtmP11lTFhEG984Ov2ldgjykqjHqfhhaiuLpdUIhcQ35lWVV4xcsv3v+/FsMfX1+2PRwUnlm4FQhMeopl056aDuVrRVklry4tA+D8ez8EnJvH8MyCwHfx+JQVZZXh7a0FhJB4AaG9BH+4TT00tv4Q2cJk1yaKZT/1mk+BNDCE/dSRRz15bNYhrPaPopZ8BkkFJbILN8oAqiiUOhQXk2Ujo1zRq5nu014s94/j7/6z2aDD2K+9qaSQ7ToQgXCTTH+qmChllDEQr7rDBf3lx49m94EG3vi05YiV3dov5dcm0sVTR1BR5wlPhAu5esY47n93Q4v0V5wwOrwEdyzTRvWnuHc+b65oWeNp7ktHDmfh5n2U7m85TPf/zpqE2wUlA3pS2+jj+peWt+g4vea08Rw2vC/9euRyycMfhbf/5eLDqW7wcsvMlVHprz1tPK8sLWPbvsD5fnLGRO6cvRaAB79xFMtLKzn/iGH0yHPz5nUnMWpgTw65+e3oY3xuApdNH0VFbSPbK+v51mMLAbj1C4fw0LyN7KisR1U5eGgfVu+oIpbbvzSZkoE9+WjDXop757OnupG756yLSvO7Cw/jlInFQOyO5klDevPjMybyt/+uCy+Z/tR3jqEg183FDwaWX//mcaOiPuOOcaDI2rLTul1Q+DSiEEwk8v8m9LK1qf8dbQrPxGoGXr8/Lc98jaZMlbUc61rNqe6lHO1aS6kWUUg9fanBJU3Xt1SL6EEDDeTSs6iE7bv30U8O8HkWk5sTvbSwX4X99GKv9qFYKtmqg6ia9mPu+98uCmjkT9dfw4D+w7g8YgJdpB+dPoFrT5vA2BtnsZ8+LNA+zLv+VEoG9gxPuvvasSVMHNybN1o5RixjiwrZ2MEbBiD84J7Rzc59zWkTYgaFW75waFRQ2PSHcwA48Y9zKauo465LjmDkgJ4tjhey6KbTwzW/Oy85Ap9fGXfjrKg0Xzh8GN+fMS5q24uLS5m/cS9Pf/dYvv73BQD89MyDWhy/qFceF00dEchrs6Bw0dQRzIoIVmccMjgcFM46bChnHTY0vO/goYGO9NANVOj7/OSMiQAU985nwuCmGvc5U4Yye/UudlTW41cY3q9HVFA49aDicK0jNAP++HFF4f3Ng8I3pkcX6M3l57j4/KFDWFFWGQ4KJ00ojkrT/G8wVvPRrecfytePjX+uVOl2QaGjPNn2kNoOaMudR0e+dx9qmOFaygRXGee7/he+g1/rH87bvmnUk0elFrKf3uzQAWzXgSz2HxTVmfrM2cfytWAh04dqDpJSAHYwkAotpIYCFCGyCv/g6Km89UFgmYi7+jUVJLH4tWV7bY478R9rIn175rb5M23RPI+tCRU8oRE3ib5KXk50G0as8fGxRu+EOugTXat4TTcukai75bauwRRPrssVzptfleZzwVJ9k5QbPEEyRw2dOlYW0tn/4WhQEJGzgLsBN/B3Vb292f584ElgKrAXuERVNzuZp3j8fk3YkdO8U6gz87ahoK/zJPc8K8FPIfUMlComSBmnuT7hQveH4dE483yTedDzBd7wHUsVvRIcLbYqerFIJ7XpM4n+2GMVcC2DQptOCTS1ezulraNRQt8y0fVo73j4pqCQIB9xRnf5VR17LnSOW5oGj6i2CF6prjeHf4faEGxiBeB0FjuOBQURcQP3AWcApcAiEZmpqqsikn0H2K+q40XkUuCPwCVO5SkRj99Pviv+yI7mBamqJnV3kYeHCVKGFxe15DNRSimWSgazn75Sw5TVQ7mioYb17h40kEe95sKmPjBgDPQd0aHv1ep3acNvWn3MJ0Epo2QXfanhJNennO7+mCmyAXdEE1C95vKa7wSe8Z3GRh3GAXq2L7MOt3LFuhvNdSW+W06k+R13qrX1zjZUvib6KsmMh4/12xO6jImuVbxfPb9Gfz6lNQW3K/zdNUbtsCMVhVgfbcvzD0KL68aqZaVzMqWTNYVjgPWquhFARJ4DLgAig8IFwK3B1y8B94qIqIOzSgpo4O+5f6ZYKnF/ejzXuP24xU8+Hnh/JfQayAWu9fSQRobUF8KyMnDlcI5rGXnSwIjduzjDtY0iqWSU7MI/bzXuvB6M2FXBte6N5IoXQcnDS7FUMEJ205caxsqO8Jjz5qq0Bz03+znI34grN+KrP3F/4GfvoVB8EAwYBz0HwqjjoaAP9B8Deb3Anduu3+bIJqFC6uhJPX2lhgEcYIjsp0R20VtqGSQVFL72DI/k7qC31NKLOvpJNcVUkC9Nywt84h/PE77Ps0MHUEUha/0jWK0l1JMf6/Rt4vTDRWKVO+1tPhJpKnzTMa68LUKFS6LrmVRNIcY1CxXgiYJVvELO54++g0/lXXKOS6Kaj1r+n6b2/yvcJJRE2nhNb12ipgAMB7ZFvC8Fjm0tjap6RaQSGAi0XKClg5a/+zJ95t3CZwVNWard8ho/zQ00a/hVcL33OgB3hxaVrAFeCby8J3SlVsOFwf1edeGeG1gaYAzwk2DzsUfdeMhhH70p0yI26jDm+I9ilX8UXtz0l2pKtYh1/hFU0ZMaetArP4e6hgb6UU0eXgqljiP7NzDGt5lJdesYtrmMERsX0ZNaXDH+GhvIo0Z6IsF9LvyBFnYN/kTD/1blKy78uO6GC/BzQT7hxdOaq9M89tKH3RsLGC5uGnFTrv1Yp8PZqQPYooPZq31YrSWUausrmnZUsm3nUZ9pQ4EcK23zgi3ZmkLv/KZVLnsVONun0FahuSmJvkpStd8YtaD84LbWgkooYMZvVlMK81M7F8PtEnx+xe0SegTnJIlAz9zo8xTktr9m1zOvZZ5D52pLjXFgL2dWtU2Wk0Eh1m9F89IsmTSIyJXAlQAlJSUtPpCMvMK+7Os5hh2eUWyqyeNG73c4+9Ah7Nuzk6W7PHjI4bxDi+jpP8CqDVvZ05jL0aP70y8fXPjYW93Ap9urOXtsHu9trKKWAjbpEC48tB9u9eETN/9euRdvK0saRAr9gua5XTQG79ZPmlBERa2H+RvdDOtbwK56LzXDi1jBCUSu/9nLW8Gwxk308NdQ7CkjRz241Uu+v458fx0gqESGAAERIkNDdaOfnVUNjCnuhaqwYU8N1dqTHr0HsL5K2ENf9mhftuogCgt7s7emkXMmD2HWp9HDFwf1zg8vQT1qYE8m5br5bGf05LM+BTl8Y/qo8AiZI0b2Y+m2Cv540WTyc9z86Pml/PSMidw9Zx0XTxvJZzurqPf4+eZxo2j0+rll5kru/MrhTC3pz5QRfVle2jR6LM/t4qoZ47hnzjoumTaSDburKch188H6PTz93WOZPjbwQPfQkEGAuy45nLW7qhlYmMfv3ljNeVOGMqxfj/Aomhf+33F85aH5nDZpEL2CExOvPW08S0srGdo30PH972tO5K/vrOP8I4ZRmOfmO08EVmvNdQsPf3MaI/v35O2VO6mq93D1jPHkuV28/HEpZx06hFEDe1LT6GV/rYfvnzKO7z6xmJ1V9UwZ0Zfjxg7kG9NH8dRHW3h4XtO6RnN+ekr49StXH8+qHVVMHt6XZcFr8ca1J/Lk/7YwaWhvinrl07dHIBDdfekRXPfcUp76zjHhzz/57WP49/IdFAfH1t/5lcN5YfE2vnjkcMoq6jlyZL/wCL1nvzc9arXY3154GK8v285pkwbx39Xl3HzeIS1+t+++9EieWbiVycP7cvZhQxjWr0fU/nd+cgpfvO9DXv7+8eFtf7poCi8u2cavzjuE/35WzrjiXtx96ZHcNXstRb3yOXJkP567cjplMYbENvePK46mpqHlDc5jlx/Nx1v2IyL89sLDKBnYk1MmDmJqyQDcbiE/x0Wu28VVp4zjtEmDGNq3R4yjw28vOJRDhvXl8scW8tBlU6P2PfntY3l9+XZOHF/EPz/awuzVu7j9oilAYCn2N1fs4LKI0Uq/u/AwJg/vG34/cXBvLps+isuOG8WybRWUDOjJv5fvYHlZJWcdNiThd08Vx56nICLHAbeq6ueD738BoKp/iEjzdjDNfBHJAXYCxfGaj7rC8xSMMSbdkn2egpO9YIuACSIyRkTygEuBmc3SzAS+FXz9ZeC/TvYnGGOMic+x5qNgH8EPgbcJDEl9TFVXishvgMWqOhN4FHhKRNYD+wgEDmOMMRni6DwFVZ0FzGq27eaI1/XAxU7mwRhjTPLsQarGGGPCLCgYY4wJs6BgjDEmzIKCMcaYMAsKxhhjwhybvOYUEdkNbGnnx4twYAmNLsSuT3x2feKz6xNfpq/PKFUtTpSo0wWFjhCRxcnM6Ouu7PrEZ9cnPrs+8XWW62PNR8YYY8IsKBhjjAnrbkHh4UxnIMvZ9YnPrk98dn3i6xTXp1v1KRhjjImvu9UUjDHGxNFtgoKInCUia0RkvYjckOn8ZIqIbBaRT0VkqYgsDm4bICKzRWRd8Gf/4HYRkXuC12y5iByV2dynnog8JiLlIrIiYlubr4eIfCuYfp2IfCvWuTqjVq7PrSJSFvwdWioi50Ts+0Xw+qwRkc9HbO+Sf38iMlJE5orIahFZKSLXBbd33t8hVe3y/wgs3b0BGAvkAcuAQzKdrwxdi81AUbNtfwJuCL6+Afhj8PU5wJsEHiU3HViQ6fw7cD1OBo4CVrT3egADgI3Bn/2Dr/tn+rs5eH1uBX4WI+0hwb+tfAJPqN0Q/Nvrsn9/wFDgqODr3sDa4HXotL9D3aWmcAywXlU3qmoj8BxwQYbzlE0uAJ4Ivn4CuDBi+5Ma8BHQT0SGZiKDTlHVeQSe5RGprdfj88BsVd2nqvuB2cBZzufeea1cn9ZcADynqg2quglYT+Bvr8v+/anqDlX9OPj6ALCawLPnO+3vUHcJCsOBbRHvS4PbuiMF/iMiS4LPvgYYrKo7IPBLDgwKbu+u162t16M7XqcfBps/Hgs1jdDNr4+IjAaOBBbQiX+HuktQkBjbuuuwqxNU9SjgbOAHInJynLR23aK1dj2623V6ABgHHAHsAP4S3N5tr4+I9AJeBn6kqlXxksbYllXXqLsEhVJgZMT7EcD2DOUlo1R1e/BnOfAKgar9rlCzUPBneTB5d71ubb0e3eo6qeouVfWpqh94hMDvEHTT6yMiuQQCwtOq+q/g5k77O9RdgsIiYIKIjBGRPALPgp6Z4TylnYgUikjv0GvgTGAFgWsRGu3wLeC14OuZwDeDIyamA5WhKnEX19br8TZwpoj0DzalnBnc1iU161f6IoHfIQhcn0tFJF9ExgATgIV04b8/ERECz5pfrap3RuzqvL9Dme69T9c/Ar3+awmMgrgp0/nJ0DUYS2DkxzJgZeg6AAOBOcC64M8Bwe0C3Be8Zp8C0zL9HRy4Js8SaALxELhb+057rgfwbQIdq+uBKzL9vRy+Pk8Fv/9yAoXc0Ij0NwWvzxrg7IjtXfLvDziRQDPPcmBp8N85nfl3yGY0G2OMCesuzUfGGGOSYEHBGGNMmAUFY4wxYRYUjDHGhFlQMMYYE2ZBwXQbIuKLWNlzaaLVOkXkKhH5ZgrOu1lEitrxuc8HVyTtLyKzOpoPY5KRk+kMGJNGdap6RLKJVfVBJzOThJOAuQRWKv0ww3kx3YQFBdPtichm4Hng1OCmr6nqehG5FahW1T+LyLXAVYAXWKWql4rIAOAxApMCa4ErVXW5iAwkMOmrmMCMXok41zeAawksIb0AuFpVfc3ycwnwi+BxLwAGA1Uicqyqnu/ENTAmxJqPTHfSo1nz0SUR+6pU9RjgXuCvMT57A3Ckqk4hEBwAfg18Etx2I/BkcPstwAeqeiSBGb8lACJyMHAJgUUJjwB8wNebn0hVn6fpGQaTCSwjcaQFBJMOVlMw3Um85qNnI37eFWP/cuBpEXkVeDW47UTgIgBV/a+IDBSRvgSae74U3P6GiOwPpv8cMBVYFFgyhx40LZTW3AQCSyEA9NTAWv3GOM6CgjEB2srrkHMJFPbnA78SkUOJv9xxrGMI8ISq/iJeRiTwmNQiIEdEVgFDRWQpcI2qvh//axjTMdZ8ZEzAJRE/50fuEBEXMFJV5wI/B/oBvYB5BJt/RGQGsEcDa+lHbj+bwOMVIbAw2pdFZFBw3wARGdU8I6o6DXiDQH/CnwgsIHeEBQSTDlZTMN1Jj+Add8hbqhoalpovIgsI3Ch9tdnn3MA/g01DAtylqhXBjujHRWQ5gY7m0FLJvwaeFZGPgfeArQCqukpEfkngyXcuAiuP/gDYEiOvRxHokL4auDPGfmMcYaukmm4vOPpomqruyXRejMk0az4yxhgTZjUFY4wxYVZTMMYYE2ZBwRhjTJgFBWOMMWEWFIwxxoRZUDDGGBNmQcEYY0zY/wdTuBh8QUE9TgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa6b242ed30>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Use active_session() on Udacity workspace\n",
    "from workspace_utils import active_session\n",
    "\n",
    "with active_session():\n",
    "    # do long-running work here\n",
    "    scores, scores_avg = MADDPG_Training()\n",
    "    with open(\"scores_file.json\", \"w\") as write_file:\n",
    "        json.dump((scores, scores_avg), write_file)\n",
    "    fig = plt.figure()\n",
    "    ax = fig.add_subplot(111)\n",
    "    plt.plot(np.arange(1, len(scores)+1), scores)\n",
    "    plt.plot(np.arange(1, len(scores)+1), scores_avg)\n",
    "    plt.ylabel('Score')\n",
    "    plt.xlabel('Episode #')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd8W9X5+PHPI3klzo6dHWeHMBJGwl6BAmUVaCmFDlro4EcH0MW3FFqgg5aWFgqFFiilBcooo0AoYYQ0EEbIguyQvewMZ9mOt6T7/P7QiGTLkmzrSh7P+/XKy9K9R/ce3djnuWfcc0RVMcYYYwA82c6AMcaYjsOCgjHGmAgLCsYYYyIsKBhjjImwoGCMMSbCgoIxxpgICwrGGGMiLCgYY4yJsKBgjDEmIifbGWitoqIiHT16dLazYYwxncrixYv3qGpxsnSdLiiMHj2aRYsWZTsbxhjTqYjIllTSWfORMcaYCAsKxhhjIiwoGGOMibCgYIwxJsKCgjHGmAgLCsYYYyIsKBhjjImwoGCMMR3AstIKlpVWZDsbne/hNWOM6Youuv99ADbfeUFW82E1BWOMMREWFIwxxkRYUDDGGBNhQcEYY0yEBQVjjDERFhSMMaYd6n0B6n0BACpqG1P6THlVPTsq66iq97F1by2Vtb7IvuoGf+S4dY2ByOudlfWoappz35wNSTXGmHY47NbXERHu/sKR3PDMEmZ872SmjOjXYvqFm/dx2YPzWtx/xG1vsPnOCzj2jrc4UO9n850XMOnnrwPwk3Mn8e3p49L+HaJZTcEYY9rBUQg4yvvr9wCwekdVwvRLt6X2gNqBen+zbW+t3tX6DLaSBQVjjDERFhSMMcZEuBYURGSkiMwRkVUislJEboiTZrqIVIrIktC/W93KjzHGZEIG+oJd5WZHsx/4kap+JCK9gcUiMktVVzVJ966qXuhiPowxxqTItZqCqu5Q1Y9Crw8Aq4Hhbp3PGGM6ApFs56B9MtKnICKjgaOB+XF2nygiS0XkNRE5PBP5McZ0X798ZRV3vfEJa3cd4Mw/vp3yswWZtHlPTdbO7fpzCiLSC3gB+L6qNh2r9REwSlWrReR84CVgQpxjXANcA1BSUuJyjo0xXdmj728CYOu+OjburuGdtbu5+Kj0NWKko0/h7+9tav9B2sjVmoKI5BIMCE+q6n+a7lfVKlWtDr2eCeSKSFGcdA+r6jRVnVZcXOxmlo0xpk2ETt5uFOLm6CMB/g6sVtW7W0gzJJQOETkulJ+9buXJGGPCwkV4RxwtlM1+CTebj04GrgSWi8iS0LabgRIAVX0Q+DzwbRHxA3XAFZqJyT2MMSbNlK5RdLkWFFT1PUhcn1LV+4H73cqDMcZkmo0+MsaYTihceKf7Dt/Nto5MxBsLCsaYbindfQrp7Ghu6UiZaKCyoGCMMSbCgoIxxqRBJjqarfnIGGNcEhoN3yGHpGaTBQVjTLcU6VNI2/HS2KeQxSFMFhSMMcZEWFAwxhgTYUHBGNM9hZ9TsE6FGBYUjDHdUleZwC7dLCgYY0wHk82pMiwoGGO6NWs8imVBwRjTLUm6x6Sm8XDZbNqyoGCM6ZbSXex29tlRwywoGGNMGnSVQUwWFIwx3Vq65yzq7BUGCwrGmG7JreaetPQp2OgjY4zJjrStp5CBgjwT57CgYIzpluzhtfgsKBhjurV09ShYR7MxxnRiHXkIaTazZkHBGNOtdaY+hUywoGCM6Za6SiGebhYUjDHdWibWVu5MLCgYY7qpjltVsOcUjDEmSzriqCFbo9kYYzLM+hTis6BgjDEmwrWgICIjRWSOiKwSkZUickOcNCIi94nIehFZJiLHuJUfY4yJpwO2HmVVjovH9gM/UtWPRKQ3sFhEZqnqqqg05wETQv+OB/4a+mmMMa6KtB51wE6FLvnwmqruUNWPQq8PAKuB4U2SXQw8rkEfAv1EZKhbeTLGmLCO3KdQVe/L2rkz0qcgIqOBo4H5TXYNB7ZFvS+leeBARK4RkUUismj37t1uZdMYYzqEpxccLBb3VjdEXmdiEj/Xg4KI9AJeAL6vqlVtOYaqPqyq01R1WnFxcXozaIzp1jpe41GsyrqDtYZMPGjnalAQkVyCAeFJVf1PnCRlwMio9yNC24wxxlXhu+50dyl0wC6KVnFz9JEAfwdWq+rdLSSbAXw1NArpBKBSVXe4lSdjjHFLRhbZyUDzkZujj04GrgSWi8iS0LabgRIAVX0QmAmcD6wHaoGrXcyPMca4xq0aQqYrHq4FBVV9jyQjq1RVge+6lQdjjGlJ+M5e01yad+RRTamwJ5qNMd1SuOxO9524m30Knb6j2RhjujJfwIm8fmbhtgQp2y7THdcWFIwxpo2q6jLxkNnBqNAlnlMwxpiOKDw9dWcfQppuFhSMMd1ae2JCNtc9cIsFBWOM6cCsT8EYY0zWWFAwxnRLbj2n0NlZUDDGdEtPzt/a7mO0pUdh9uryVqXPdMiyoGCM6ZYa/U7yRC6Yt3FvVs6bKgsKxhhjIiwoGGNMB2ajj4wxphPLxPxEbrKgYIwxbZSJZ9cyHWQsKBhjjImwoGCMMSkqP1DPvppGADbtqaEx0HwEUyYmrXOTmyuvGWNMl3LcHbMBWHrrOZzxh7c569DBrp8zpqM5A/HGagrGGNNKNY1+AOZt2NNsn3U0G2OM6TIsKBhjTCtF5k3KwLnsOQVjjOkk4jXxp7uj2YakGmNMJxGvuLY+BWOM6WbCtYGuOOu2BQVjTKuddfc7/OKVldnORsYc/cs3eWDO+sj7E347O2PnvuC+9zJ2LrCgYIxpg/Xl1fzj/c3ZzkbG7K/1cdcba5pt7+xNRfFYUDDGmDay5iNjjDFdmmtBQUQeFZFyEVnRwv7pIlIpIktC/251Ky/GGOOGLlhRcHXuo38C9wOPJ0jzrqpe6GIejDEmoXY1AXXBqOBaTUFV5wL73Dq+McaY9Es5KIjIKSJydeh1sYiMScP5TxSRpSLymogcnobjGWO6uZeXlPHykjLum70OTaEaEL1QTun+Wh6Ysz6lz0HrRh+VH6jnT2+tTTl9PJmYlDul5iMRuQ2YBhwC/APIBf4FnNyOc38EjFLVahE5H3gJmNDC+a8BrgEoKSlpxymNMV3dDc8sibw+c9IgjhjeN2H66PL/m48t4pOdB/jMlGGUDOyZ1nz96NmlvLuu+ayqrZGJ1qpUawqfBS4CagBUdTvQuz0nVtUqVa0OvZ4J5IpIUQtpH1bVaao6rbi4uD2nNcZ0I63tLwhPiZ1qDaA1x6/3BVqXmSxJNSg0arA+pQAiUtjeE4vIEJFgxU1EjgvlZW97j2uMMZkSd+4jF2/nO0zzEfCsiDwE9BORbwFfB/6W6AMi8jQwHSgSkVLgNoLNTqjqg8DngW+LiB+oA67QVBvyjDEmTaRzr56ZdikFBVX9g4icDVQR7Fe4VVVnJfnMF5Psv5/gkFVjjMmaeLeiqU5/3RXvY5MGBRHxAm+p6hlAwkBgjDHdXWeveSTtU1DVAOCISOIufGOM6SIWbN6HP+AkTZfpPoVMSLVPoRpYLiKzCI1AAlDV613JlTHGZNGPn1vK+vJqbjpvUsJ0nT0AxJNqUPhP6J8xxnRZ0YX88rIK147dkaXa0fyYiOQBE0Ob1qiqz71sGWNM5nWWgttNqT7RPB14DNhMcKjsSBH5Wmh+I2OMMV1Eqs1HfwTOUdU1ACIyEXgamOpWxowxJtOih5h211pDqk8054YDAoCqriX0IJoxxnQV3TQOxEi1prBIRB4hOAkewJeBRe5kyRhjsiO6dmA1hcS+DawCrg/9WxXaZowxndodM1fz+b9+ALRuKuxUTL9rDtc9/XFaj+m2VINCDnCvqn5OVT8H3Ad43cuWMcZkzqIt+4H01w42763llaXb03tQl6UaFGYDPaLe9wDeSn92jDEmezp6i1EmptBINSgUhNc+AAi9Tu8KFMYYk2UxfQodMERkop8j1aBQIyLHhN+IyDSC010bY0wXYkNSUx199H3gOREJN44NBS53J0vGGJMd6QgELR0iHTEm681HInKsiAxR1YXAJODfgA94HdjkfvaMMSZzumnlIEay5qOHgMbQ6xOBm4EHgP3Awy7myxhjMmrehr1pWTSnky+nkDQoeFV1X+j15cDDqvqCqv4cGO9u1owxJnO++LcPY2oKHbHWUNy7wPVzJA0KIhLud/gU8L+ofan2RxhjTKcQr6LQ2tqDm8Hk6pNHu3j0oGQF+9PAOyKyh+Boo3cBRGQ8UOly3owxJqM6+prLmWiaShgUVPUOEZlNcLTRm3rwinmA69zOnDHGZJK2+Kb7SNoEpKofxtm21p3sGGNMFsVtPmrdIdy8m5cMjElN9eE1Y4zp8mI7mttWVejsFQwLCsYYExKvT6EjFfKZ6FOwoGCMMSEdKQBkiwUFY4wJsUV2LCgYY0xCHWmYatbnPjLGmO7E6UABIFtcCwoi8qiIlIvIihb2i4jcJyLrRWRZ9NTcxhiTDfFCQkcKE5KBrmY3awr/BM5NsP88YELo3zXAX13MizHGJKdxX3YrrgUFVZ0L7EuQ5GLgcQ36EOgnIkPdyo8xxiTTkZuPjpPV5Fa7v95zNvsUhgPbot6XhrYZY0xCP/j3EsbfPDOltJv31FDd4E8prd+JXnlNQz9bn790ExyeyPstA1f9w/VzdYqOZhG5RkQWicii3bt3Zzs7xpgse/HjspgCPJHpf3ibKx6e53KOkmvPKKYiKskXP77eI9OYo/iyGRTKgOhvOCK0rRlVfVhVp6nqtOLi4oxkzhjTdawoq8p2Ftrl6zmvA9DYy/3GlGwGhRnAV0OjkE4AKlV1RxbzY4wxERr5mf32o0u87wNQO+R418/l2kI5IvI0MB0oEpFS4DYgF0BVHwRmAucD64Fa4Gq38mKMMRmT1k4I5Tc5f2eo7GNWYCrD8nul8djxuRYUVPWLSfYr8F23zm+MMemQrY7mAhr4pODgvfIv/FfyUAbO2yk6mo0xJtPaHAzSNBfF0Z71kdd/9X+GUh2UluMmY+ssG2M6lDdX7mRfTSNXHFcS2fb84lIK87ycN7l9jzJlah6jJ+ZtZuSAnu06xmmeZQD8zHc1zwdOAzLzRLMFBWNMh3LNE4sBYoLCj59bCsDmOy9o17FL99elnLbN4UOVn7+8sq2fjjhKNgDwr8BZZGYlhSBrPjLGmA4mn0ZO9K5iTuBIogOCzZJqjDFZlo2O5uM8nwAw28n8PKEWFIwxJp3aeTvfg3qeyLsTgIXOIenIUatYUDDGmHjaWkVoZ9Xi3twHIq+36OCYfdZ8ZIwxWZKNJ5oHUsmnPB/xoXMox9ffTz35GTt3mAUFY0xa1DUGmLdhb7uOEW/IaEVtY5uP9+HGg/nZXlHHJzsPtPlYqfrDm2ubbXt7TXlKoWW6ZyleUX7lu5JdDGi234akGmM6jVteXM5/Pi7jnRunM2pgYZuO8dSCrTHv630BjvrlrDbn6Y6ZqyOvT7rzf206RmtbgyrrfM22XfWPhSl9doKnjAbNYZWWJE/sEqspGGPSInwXfqA+tbUL4tm2L/Y5gifnb20hZVekHONZy04dgLZQNFufgjGmW2nafu8POInTuzheNNNDUU/2rOA4zxpqKcjsiZuwoGCM6Tg04dvmyTNQcGcqNpRIOQC/8n8lQ2eMz4KCMSYt3GjaSFboZ3+lg/Q5VLbSoDksdia2mCYTk11YUDDGdBitLeQdN5uPMhxyjvRsYL5zKA3kZfS8TVlQMMZ0GK3tIwg4SqM/cb9De2VqZtViqWCnNh+GGi0THc02JNWYbq78QD3H3TGbuz4/hcumtX9h+HAZuqKskgv//F5k+8TBvXjzB6cDMOeTcq7+50KOGN6HFWVVkdlPm5a/v3v9k5j3763bw4LN+yLvP//gB9Q0BJjz4+nsqKzjxN+2bdhpPCvKqnhhcSnnHD44eeJ2yMFPT+oZJvvYTV9Xz5UKqykY081t3F0DwHOLS9t1nKZ3se+v3xPzfu2u6sjrmcuDy7GvKKtq1TlmrdrJ3+ZujLxfUVbFpj3B/K9x4cG0/3xc6noj0i05T7Ks4BoANmn71otIBwsKxpi0aE0rS443fjtIewpgNwpvr8f9IjK8mA7AR86EJKndbz+yoGCMyTivp4WgkKRkF5EW29XdaPvPaSGf6XKobGGcZwebnMFc2PBrNuowV8+XCgsKxpi0aE0naI4Ld+Bu9Ad7PeLqsxCv5f8UgFwJsELHJk1vTzQbYzqdVIZytlhTaEcjkBuFt9s1hbArG3+akfOkwkYfGWMyrqXCNrpgb21zkDt9CuLSE3LKU7l3APB3/3kpdzDbw2vGdFG+gMP/Pb+UrXtrk6Ytr6rnh/9eQr0v0GxfZa2PG575mAP1zWfm7Ij+8vZ6HpiznveajEyK59XQCKVoOyrrqG1sfh2WlVZwx6ur0pLHaP9dtoONe6qTJ2ylEbKHk7zB/FZpz7Qfvz0sKBiTBYu37OfZRaX8+PmlSdP++tXV/OfjMl5fsbPZvgfnbuDlJdt5fN6WuJ+t9wUYfdOrPPbB5vZmOalU5vr//etruOuNNazcnnwo6vee+rjZtjdW7oqb9rIH57E5hQDbFp/9ywdpP2Z/Dg6f3U2/tB+/PSwoGNPBhVsv4nUyhje11NRSURusQfzl7fXpz5gL2jqCqLPNgVQklQDMChzDi4GTU/6cZKCn2YKCMR1cooIyXEZkeppnt7T5a3Sy7z9agrW+W3zfoC7LU2U3ZUHBmA4uUXmXrMkm05O6ZYubE+O54VTPcsp0IOX0b9XnOn1Hs4icKyJrRGS9iNwUZ/9VIrJbRJaE/n3TzfwY0ymFyrtETQfJisRMrO0b1p7yua2f7UwhYTD7ONO7hIXOIdnOSlyuDUkVES/wAHA2UAosFJEZqtp0iMC/VfV7buXDmK4iXrHeEZuP2jdVRRv7FDrSBUjiQu88AF4JnJjlnMTn5nMKxwHrVXUjgIg8A1wMpH/cmDEdTOn+WvZWN3LkyPaPLAkXlPEqCltCI27eXlvODWc1nzcnWVn54ca97KtpbHPelpVWsGDTPr556sGncRds2svK7ZWtKt437K7GF3CYv3Ff8sRxOB0kJkyUbVzgnc8z/jPYwUB+nfN3aimgQgsZJBVc6P2QIgmOvNqsQ1p9/M4+dfZwYFvU+1Lg+DjpLhWR04C1wA9UdVvTBCJyDXANQElJiQtZNSa9TvndHIDIlNDtES7Y4zUBzVi6HYCPt1awcXc1Y4t7xT1GvMJEVbni4Q/blbeL7n8fgAunDIuc4zczg9NdlwxIffz9p/74TrvykW1eAoyWnbyZ/xMAbsj5Dy8HTuJib8vDWdsSFDIh2x3NrwCjVXUKMAt4LF4iVX1YVaep6rTi4uKMZtCYbIsEhSR3idUN/lYdN5131/EerNtZVZ++E2TYOCkjn2ANyoODlwDnehbQj/jTc/9fzjPMzr8xZltLAeHf/ulMqf8bAbytzlcm+obcrCmUAdErdowIbYtQ1b1Rbx8Bfu9ifozplCLNR236bMsCbYwK8drvM9Gs4bY8fJzsWcE/8u5KmG50/VPNtn3Re3Bxn2Pr/0I9eVzsfZ8NOoxlzlhqyWes7CAfH6t0dLqznlZuBoWFwAQRGUMwGFwBfCk6gYgMVdXws+wXAatdzI8xnVKqNYVE4n20rUEhHk+njArKnTl/Y4xnJ8d7PkmePOTtvB/wFd/NlGqw1eLTnoX0kToA5gYmh1ZPE/4VODvmcxt0eNpy7ibXgoKq+kXke8AbgBd4VFVXisgvgUWqOgO4XkQuAvzAPuAqt/JjTGd1sOhOb8EbaOvTwy18rLOFhUNlK1fkvB133+j6JxkhezjXs4A3nGk8nHsP851JXJXzJqM9uzjXs4BHAsH+ou/mvATADY3f4WXnFFfz3Nk7mlHVmcDMJttujXr9U6DjzBlrTAeUak2htWV8OmsKnbGiMEJ2R16vdkq4svGn9JB6yrU/IJRqcaTgP6/xTvpQQyH1XJYzl5/lPsm7zmTWaAl9qeGlwEmuB4RMsamzjUlAVQk4So43/piMgKN4JPjT65E2z02T+DzBwttxDv5UmrftK+APODHHaJrGcTTy9G/ToOA4iuMonqhprf0Bp9n38jlOsxyKCP6mQaaDDBNtyU05TwMwrf6v7KFvcKP2bTF9FYXc6L+Wy3LmAnBNzqu8G5jMKE85s/xTXc9vpmR79JExHdqf3lrH+Fteoy7OdM0A426eybceX8T4W17jkXc3tfk8t7y0gvG3vBZ3X7hc//aTH7F21wFO/8Mcxt08s1n6p+dvZfwtr7EratTPwVqGsPtAA6fdNYfxt7zG+Fteazbd9qIt+7nwz+9F3lfV+xh/y2v85e0NkW3ry6s55GevN8vjtU8sbjbzaWOgefDoGJQf5jzLOE+wO3MvvVv16ePqHwDgUu+7/CnvLwApr4fQGVhQMCaBpxZsBUi4XsFbq8sBeH5xadvPMz94nngje6K3LCutZNu+urjHeHFJcHDflhamkJ5+1xxK9x/8bLyH1lbtOFiw7z7QAMR+rxVllXGPvbyF7R3NGNnBC3m3c32oH2Bq/V/RVhaDTecrWuGM5snAp9KWx0Q6fZ+CMZ1duIxOpSUkHX+wwSak2ANFB4qE0zlo8zTRyWua1HaS9SmEjxOdm3T2Q2RCMfu5K/dh/hk4hz7Ucl/eA5F91zd+l7203FyUSJX2oI/UMb3hj2zuQrUEsKBgjKtUtVX9DH5HyWnyTFOqxXC8eYMSzR7arA+g2WeDP6Oz39YRS5l2iee9SNMOwHRv7GJGq52RzHBSX8egqSkNf2/zZzs6CwrGJBAuENtaCXAUvK34sC/gUJAbGxWiy+FEASb4tGtsEEpUhDtJawrBn9HPIHTUiefy8NFILgD35f6Zi0KTzgFUak9qKGAI+/my72ZqNZ9tOihbWW2XTCyyY0HBGBeFRyWlyh9I3KeQqFCOV1NIlD7ZXX/4eNFBoaP1HQsOo2QXM/Nupqc0xOybG5jMd303cICOtQZyR2dBwRgXpbr4i0eCtYp4wz3bc3eeqDKQrH8gnJXom9OOtpjNz3P+xddzYkdD+dTLtb7vM9vpOsNEwzLxOIgFBdNlPPLuRnK9Hr520uiMnG9vdeyd6Sc7D3Djc0u567IjI9sCjrK9oo6bX1zO/V86htoGP19/bCEryoKjfNbuOsDlD82LFN7+gPKLV1ayansVF0wZyqXHjODddXsix7vx+WUt5scXqmV84aF5fPOUMYwuKuSEsQMAKKtoPmLpqn8sjHuct9eUM/2QQZEAUNsYYPJtbzB1dH/OnNQxml08OHzP+1IkIHy24Rd8rOMRFA/apsnmTJAFBdNl/PrV4NRZbgSF6PvjxVv2celf58VN99zi0piCM6DK3bPW8vaa3cxcvoNV26siAQGgotbH/E0H1xAIOMo/3t8MwPxN+yjMa9uf6CPvtf2Ziav+sTBmyu+t+0JrNqzZzdtrdrf0sYy62vsaP8x9HoAT6v/MTgYCoAjxnygxqbKgYEwKoptNws8ltOTfiw4uCdK0M7e1/YSt6Y9INzeaioawlz30xZ+k6PHg4DR5fuBq72vclvtEzLYLGn4TCQhdRc88L7UtPCxpzykY00FEl4/Jysrotvqm7fbJ5sNvWhBnc06hdD6S0Ica/pt3MyWe3bwfOJwv+25pMe2x8gnP5f+Szc5g/ha4gH5UIyg/zn0uJt0Njd9hZQefhrotsj2NlAUFY1LQmrvm6BFETUf4JCvkmwaRrlBTyKeRj/L/HzkS7Lk+2buSqf41TPWsZZmO41j5hFIt5i1nKud55/P73L8BMNqzizs8j8Yc6/yG3zBM9tKHmi4zAV1TyYcdu8uCgjEpiH1WIHHa6II9zmCihHxNhqRmc52CZM8xJJKHDw8OX/T+L9Lk84T/LF53juXJvN/yQv4vEn7+h43XskzHckvOk5wRevDsO43Xs0pHd/hFatrLagrGdAKtaT7yR0WCZjWFJOdp9MdGkWwGhdZOaeHB4TLvO9ya8ziFTZ4Z+Kf/HG73XwXA//m+xZGykSU6jpFSzm7txzTPWo6WdTh4uM1/Fe84wRFcV/t+Ai1PO9U1Jfgvtz6FLmrr3lrKD9QzbfSAbGclLVL9PuvLD7B0WyVjiws5uqR/wrTt8fHW/XzvqY+5+uTRnH3YYJaWVlLb4OeK40oiaZZsq6BXfg7jBx1c6H7znhr2VDdQVlHH+ZOH8u663ZFJ4V5Ztp0Txg5gQGE+765LPALno60VkddvrdrFrFW7AFi8eT/7a5tPQhdt1qqdMe+v/dfi1L50mh3/m7fYVdWQPGFIIXXclvM4X8h5J7Lt3cARrNZR3OO/lDoKItufDZzBs5wR8/knAue0P9NdRLZrCtJRH1tvybRp03TRokXZzka7jL7pVYCYYX+dWarfJ5wO4LlrT+TYOEGkwR/g1pdW0q9nLieMG8gZh7Q8Ln7Wql2U7a/lqpPHsLOynhN+Ozvh+dffcR4/eHYp8zfupTxU2Bf1ymdPdQMTB/di7a7qSNpvnDKGv7djWGe2eUJP+m7XgTSQ58o5BIcveN/hd6E+AIC3A0fydd+NzUYOmdT1Kcihqt4fd9+Cmz/FoD4FcfclIyKLVXVasnRWUzBZcdmD8+IGkddX7IwM6Xxo7saEgeZbjwdvDq46eQy3vrwi6TkXbN7HK0u3x2zbE3oALTogAGyP87BXx6UcI+sYKeUc4dnMaZ5ljJUd5EpwWONu7cs6Zzi76cdO7U+lFgLCB87hLNHxbTrjhZ553J/358j7vdqbe/yf51+Bs8j+vW5qLpgylFeX7UieMAUbf3M+x97xFnvjTEeeTJ7XwyNfm0ZRr3zOv+/dhIMLWlrsKZ0sKJguIaWRMp2rUpxUPo38MuefXN5kneG92puXnZPZo3242vs6xVJJsbeSUi1ihOxpdpxNzmB6ST0vBE6lSgvZpENwEHIJMNeZTBW9EBzGyg5qtIDDPFsiAeGv/s/wD/+5zdYYyISlt53Dkb94k975ORxoiH9nnci3Th2btqDgaccose+cMY7TJhazKrRIUaKgkInRaBYUjImjo83x04caLva+z+mepeThp4/UcJRnY2T/a4FjmRFw6fW7AAAWSElEQVQ4ibU6gg06PLL9Yf+FFNDIdooAmCIbGCAHGCnl3JjzLH2klnryGCO7uDbnv3HPXaYD6UEDAyS2NvWQ/wJ+5/+iC982Ne3tdE33/3Fb89PsWZYEB8ptzZS7bWRBwXQbrVkLoL2zgRZSx1Ge9SxwDsWHl3hNKh6c4FrLofb3fBpRBD/eqDZ55Rvemfwo5/mYWUD3ay/2ay9+4/8SLwROa7ENfx99Yt4v03GRGtMTgbNDWyVy/j7UMN6znSmyET9eDpFt5IqfRs2lWCqopYAGcljoTOKFwGltvTwdQkfpT20+DLnltDkeaz4y3Yyb88XHm5a6JcnuIvPw4cfLl7yzGS9l9JUaVjijOcvzMUd6NsQU4AEVGshjnQ6njnzy8JFDgCmexB3ZfvVEHvja6AzhR43fZpmOJR8ftbStszFW7LVuII/d5LHb6c88Dk/D8Tu2jrKInD90BxL+1U80DDnHmo/cUdcYwO849C7IxR9w8Dsas7BJgz9Avc+hT0FOpJByHKXB79Ajz0tFbSN1vgB9CnIpzD94CavqfeR5PQRCx6sOjyAQqG7w4xWJqWLW+wJU1ProVZBDr/wcVJU6X4CebZwELVUBR2kMfRcIrj/s9Qh5Xg9V9X7ycjz4/A6+gEO/nnk4Gvw+qkpFrY+8HE9k1a4G/8E5WsLDN+saA/TtmYvXIxyo91FR66N3QfPvtKKskl75OeR4Ba9HqPc5rNt1ICbNpj015HolcjfV6HcY3Cc/Zjz/+vIDrC+PbdqI56Ot+wEYyl6Gyl6KpZJ8GhnvKcODMlZ2MEm2stiZyNIdJ/EV7y76UMMJntUUSCMOHoqpQFDGenY2O/5nve9Trv34d2A6HhxyCVCmRRRII4XUc5hnC32pZozspEB8rHRGsUGH4SOHIiqppoA1TgmnepfRqLn48eLBYbZzDP8MfJpwIV5rM4ACrVsqNZ72PJyXTk1XwEsUFNrTd5GqbhcU9lY3MPXXbwHw6vWncM+stby1upwXv3NSZOz8IT8LTsd703mTuPb0cQDcNmMlT3y4hf9edwoX/vm9yPHCo2O27avl1N/PaVVeJv384Dzw/73uFGavLueet9Yy43snk5/j5ZAhvdv+RRP4+csreGr+Vjb99nw+2XmA8+59Fzg4PDOezXdewA3PLGFGk9E70Y69461W5SP6OrbkjD+8nTTNWXfPbbbNg8NY2U4uAbwEOMWzguFzd7IgfwmDpCLOUaBCC9muRVyWM5fLGuYSWsiLA9qDdTocH8LW0Ipdi/0TKacf653hvOdMZrjsYaf2Zy998bXzz+q+wOfa9fnuIi80EufY0f2Z04bZW9MVEwb1zg+9aluBPaaoMOZ9BlqIEup2QSF6DvmVZVWRGS8/+5cPmg1/fGXp9khQeGrBViD40FM8m/bUtCtfK8oqeWlJGQAX3f8+4N5zDE/ND34XX0BZUVYZ2d5SQAhLFBDaSnAiberhsfWHyRYmezZRLPup13wKpIEh7KeOPOrJY7MOYbUzilryGSQVlMguvCgDqKJQ6lA8TJaNjPLEzma6T3uxzBnHI855bNBh7NfeVFLIdh2IQKRJpj9VTJQyyhiIX72Rgv6qk0az+0ADry5vPmJlt/ZL+7WJdtnUEVTU+SIPwoV9Z/o4/vL2hmbprz55dGQK7nimjepPce98XlvRvMbT1OeOHs6Czfso3d98mO5Pzp2E1wMlA3pS2xjgxueXNes4ve7M8RwxvC/9euRy+cMfRrb/8bIjqW7wc9uMlTHprz9zPC8uKWPbvuD5fnj2RO6etRaAB79yDMtKK7noqGH0yPPy2g2nMmpgTw679Y3YY3xqAleeMIqK2ka2V9bztUcXAHD7Zw7jobkb2VFZj6py6NA+rN5RRTx3fm4yJQN78uGGvRT3zmdPdSP3zl4Xk+bXlxzB6ROLgfgdzZOG9OYHZ0/kz/9bF5ky/YlvHEdBrpfLHgxOv/7VE0fFfMYb50DRtWW3dbugsDyqEEwm+v8m/LKlR//b2xSejdkM/I6TkTVfYylTZS3He1ZzhncJx3rWUqpFFFJPX2rwyMHrW6pF9KCBBnLpWVTC9t376CcH+DSLyM2JnVrYUWE/vdirfSiWSrbqIKqm/YAHPthFAY38/sbrGNB/GFdFPUAX7ftnTeD6Mycw9uaZ7KcP87UPc288g5KBPSMP3X3p+BImDu7Nqy0cI56xRYVsbOcNAxBZuGd0k3Nfd+aEuEHhts8cHhMUNv32fABO+d0cyirquOfyoxg5oGez44UtvOWsSM3v7suPIuAo426eGZPmM0cO49vTx8Vse25RKfM27uXJbx7Plx+ZD8CPzjmk2fGLeuVx6dQRwbw2CQqXTh3BzKhgdfZhgyNB4dwjhnLuEUMj+w4dGuxID99Ahb/PD8+eCEBx73wmDD5Y4z5/ylBmrd7Fjsp6HIXh/XrEBIUzDimO1DrCT8CfNK4osr9pUPjKCbEFelP5OR4+ffgQVpRVRoLCqROKY9I0/RuM13x0+0WH8+XjE58rXbpdUGgvX0dbpLYdWnPn0Z7v3YcapnuWMMFTxkWeDyJ38Gud4bwRmEY9eVRqIfvpzQ4dwHYdyCLnkJjO1KfOO54vhQqZPlRziJQCsIOBVGghNRSgCNFV+AdHT+X194LTRNzT72BBEo+jzdtrc7zJ/1iT6dszt9WfaY2meWxJuOAJj7hJ9lXycmLbMOKNj483eifcQZ/sWiVquvGIxNwtt3YOpkRyPZ5I3hxVmj4Llu6bpNzQCVI5avjU8bKQyf4PV4OCiJwL3At4gUdU9c4m+/OBx4GpwF7gclXd7GaeEnEcTdqR07RTqDPzt6Kgr/Oltp6V4FBIPQOliglSxpmej7nE+35kNM7cwGQe9H2GVwPHU0WvJEeLr4peLNRJrfpMsj/2eAVc86DQqlMCB9u93dLa0Sjhb5nserR1PPzBoJAkHwlGdzmqrq0LneOVg4NHVJsFr3TXmyO/Q60INvECcCaLHdeCgoh4gQeAs4FSYKGIzFDVVVHJvgHsV9XxInIF8DvgcrfylIzPccj3JB7Z0bQgVdWU7i7y8DFByvDjoZZ8JkopxVLJYPbTV2qYsnooVzfUsN7bgwbyqNdc2NQHBoyBviPa9b1a/C6t+E2rj7sSlDJKdtGXGk71LOcs70dMkQ14o5qA6jWXlwMn81TgTDbqMA7Qs22ZdbmVK97daK4n+d1yMk3vuNOttXe24fI12VdJZTx8vN+e8GVMdq0S/eo5Gvv5tNYUvJ7Id9c4tcP2VBTifbQ16x+EJ9eNV8vK5MOUbtYUjgPWq+pGABF5BrgYiA4KFwO3h14/D9wvIqIuPlVSQAOP5P6BYqnEu/wkrvM6eMUhHx+8uxJ6DeRiz3p6SCND6gthaRl4cjjfs5Q8aWDE7l2c7dlGkVQySnbhzF2NN68HI3ZVcL13I7niR1Dy8FMsFYyQ3fSlhrGyIzLmvKkq7UHPzQ6HOI14cqO++mN/Cf7sPRSKD4EB46DnQBh1EhT0gf5jIK8XeHPb9Nsc3SRUSB09qaev1DCAAwyR/ZTILnpLLYOkgsKXn+JvuTvoLbX0oo5+Uk0xFeTLwekFPnbG81jg0+zQAVRRyFpnBKu1hHry452+VdxeXCReudPW5iORg4VvJsaVt0a4cEl2PVOqKcS5ZuECPFmwSlTIBZzYO/h03iXneCSm+aj5/2l6/78iTUIppE3U9NYlagrAcGBb1PtS4PiW0qiqX0QqgYFA8wla2mnZ2y/QZ+5tfFJwMEu1W17mR7nBZg1HBc87rwBwb3hSyRrgxeDL+8JXajVcEtrvVw/eOcGpAcYAPww1H/vUi48c9tGbMi1iow5jtnMMq5xR+PHSX6op1SLWOSOooic19KBXfg51DQ30o5o8/BRKHUf3b2BMYDOT6tYxbHMZIzYupCe1eOL8NTaQR430REL7PDjBFnYN/UQj/1blKx4cPPfCxThcnE9k8rSm6jSPvfRh98YChouXRryUaz/W6XB26gC26GD2ah9Wawml2vKMpu2Vatt5zGdaUSDHS9u0YEu1ptA7/+Asl70K3O1TaK3wsynJvkpKtd84taD80LaWgko4YCZuVlMK89P7LIbXIwQcxesReoSeSRKBnrmx5ynIbXvNrmde8zyHz9WaGuPAXu7MapuqTtHRLCLXANcAlJSUJEkdX15hX/b1HMMO3yg21eRxs/8bnHf4EPbt2cmSXT585HDh4UX0dA6wasNW9jTmcuzo/vTLBw8B9lY3sHx7NeeNzeOdjVXUUsAmHcIlh/fDqwEC4uW/K/fib2FKg2jhX9A8r4fG0N36qROKqKj1MW+jl2F9C9hV76dmeBErOJno+T97+SsY1riJHk4Nxb4yctSHV/3kO3XkO3WAoBIdAgREiA4N1Y0OO6saGFPcC1Vhw54aqrUnPXoPYH2VsIe+7NG+bNVBFBb2Zm9NI+dPHsLM5bHDFwf1zo9MQT1qYE8m5Xr5ZGfsw2d9CnL4ygmjIiNkjhrZjyXbKvjdpZPJz/Hy/X8v4UdnT+Te2eu4bNpIPtlZRb3P4asnjqLR73DbjJXc/YUjmVrSnykj+rKs9ODosTyvh2unj+O+2eu4fNpINuyupiDXy3vr9/DkN4/nhLHBBd3DQwYB7rn8SNbuqmZgYR6/fnU1F04ZyrB+PSKjaJ79fyfyhYfmceakQfQKPZh4/ZnjWVJaydC+wY7v/153Cn96ax0XHTWMwjwv33gsOFtrrld4+KvTGNm/J2+s3ElVvY/vTB9PntfDCx+Vcu7hQxg1sCc1jX721/r49unj+OZji9hZVc+UEX05cexAvnLCKJ74cAsPzz04r9HsH50eef3id05i1Y4qJg/vy9LQtXj1+lN4/IMtTBram6Je+fTtEQxE915xFDc8s4QnvnFc5POPf/04/rtsB8WhsfV3f+FInl20jc8ePZyyinqOHtkvMkLv6W+dEDNb7K8uOYJXlm7nzEmD+N/qcm698LBmv9v3XnE0Ty3YyuThfTnviCEM69cjZv9bPzydzz7wPi98+6TItt9fOoXnFm/j5xcexv8+KWdccS/uveJo7pm1lqJe+Rw9sh/PXHMCZXGGxDb1z6uPpaah+Q3Oo1cdy0db9iMi/OqSIygZ2JPTJw5iaskAvF4hP8dDrtfDtaeP48xJgxjat0eco8OvLj6cw4b15apHF/DQlVNj9j3+9eN5Zdl2ThlfxL8+3MKs1bu489IpQHAq9tdW7ODKqNFKv77kCCYP7xt5P3Fwb648YRRXnjiKpdsqKBnQk/8u28GyskrOPWJI0u+eLq6tpyAiJwK3q+qnQ+9/CqCqv41K80YozTwRyQF2AsWJmo+6wnoKxhiTaamup+BmL9hCYIKIjBGRPOAKYEaTNDOAr4Vefx74n5v9CcYYYxJzrfko1EfwPeANgkNSH1XVlSLyS2CRqs4A/g48ISLrgX0EA4cxxpgscbVPQVVnAjObbLs16nU9cJmbeTDGGJM6W0jVGGNMhAUFY4wxERYUjDHGRFhQMMYYE2FBwRhjTIRrD6+5RUR2A1va+PEiXJhCowux65OYXZ/E7Poklu3rM0pVi5Ml6nRBoT1EZFEqT/R1V3Z9ErPrk5hdn8Q6y/Wx5iNjjDERFhSMMcZEdLeg8HC2M9DB2fVJzK5PYnZ9EusU16db9SkYY4xJrLvVFIwxxiTQbYKCiJwrImtEZL2I3JTt/GSLiGwWkeUiskREFoW2DRCRWSKyLvSzf2i7iMh9oWu2TESOyW7u009EHhWRchFZEbWt1ddDRL4WSr9ORL4W71ydUQvX53YRKQv9Di0RkfOj9v00dH3WiMino7Z3yb8/ERkpInNEZJWIrBSRG0LbO+/vkKp2+X8Ep+7eAIwF8oClwGHZzleWrsVmoKjJtt8DN4Ve3wT8LvT6fOA1gkvJnQDMz3b+XbgepwHHACvaej2AAcDG0M/+odf9s/3dXLw+twM/jpP2sNDfVj7BFWo3hP72uuzfHzAUOCb0ujewNnQdOu3vUHepKRwHrFfVjaraCDwDXJzlPHUkFwOPhV4/BlwStf1xDfoQ6CciQ7ORQbeo6lyCa3lEa+31+DQwS1X3qep+YBZwrvu5d18L16clFwPPqGqDqm4C1hP82+uyf3+qukNVPwq9PgCsJrj2fKf9HeouQWE4sC3qfWloW3ekwJsisji09jXAYFXdEXq9Exgcet1dr1trr0d3vE7fCzV/PBpuGqGbXx8RGQ0cDcynE/8OdZegYA46RVWPAc4Dvisip0Xv1GBd1oakhdj1iOuvwDjgKGAH8MfsZif7RKQX8ALwfVWtit7X2X6HuktQKANGRr0fEdrW7ahqWehnOfAiwar9rnCzUOhneSh5d71urb0e3eo6qeouVQ2oqgP8jeDvEHTT6yMiuQQDwpOq+p/Q5k77O9RdgsJCYIKIjBGRPIJrQc/Icp4yTkQKRaR3+DVwDrCC4LUIj3b4GvBy6PUM4KuhERMnAJVRVeKurLXX4w3gHBHpH2pKOSe0rUtq0q/0WYK/QxC8PleISL6IjAEmAAvown9/IiIE15pfrap3R+3qvL9D2e69z9Q/gr3+awmOgrgl2/nJ0jUYS3Dkx1JgZfg6AAOB2cA64C1gQGi7AA+ErtlyYFq2v4ML1+Rpgk0gPoLtuN9oy/UAvk6wY3U9cHW2v5fL1+eJ0PdfRrCQGxqV/pbQ9VkDnBe1vUv+/QGnEGwaWgYsCf07vzP/DtkTzcYYYyK6S/ORMcaYFFhQMMYYE2FBwRhjTIQFBWOMMREWFIwxxkRYUDDdhogEomb2XJJstk4RuVZEvpqG824WkaI2fO7TIvKL0Iybr7U3H8akIifbGTAmg+pU9ahUE6vqg25mJgWnAnNCP9/Lcl5MN2E1BdPthe7kfy/BdSYWiMj40PbbReTHodfXh+bMXyYiz4S2DRCRl0LbPhSRKaHtA0XkzdD8+o8QfGApfK6vhM6xREQeEhFvnPxcLiJLgOuBPxGcSuJqEekSTwGbjs2CgulOejRpPro8al+lqk4G7idYEDd1E3C0qk4Brg1t+wXwcWjbzcDjoe23Ae+p6uEE55cqARCRQ4HLgZNDNZYA8OWmJ1LVfxOcbXNFKE/LQ+e+qD1f3phUWPOR6U4SNR89HfXznjj7lwFPishLwEuhbacAlwKo6v9CNYQ+BBem+Vxo+6sisj+U/lPAVGBhcMocenBworSmJhJcaAWgUINz9RvjOgsKxgRpC6/DLiBY2H8GuEVEJrfhHAI8pqo/TZgouExqEZAjIquAoaHmpOtU9d02nNeYlFnzkTFBl0f9nBe9Q0Q8wEhVnQP8BOgL9ALeJdT8IyLTgT0anEt/LvCl0PbzCC6vCMEJ0j4vIoNC+waIyKimGVHVacCrBFfp+j3BCeSOsoBgMsFqCqY76RG64w57XVXDw1L7i8gyoAH4YpPPeYF/iUhfgnf796lqhYjcDjwa+lwtB6dK/gXwtIisBD4AtgKo6ioR+RnBle88BGce/S6wJU5ejyHY0fwd4O44+41xhc2Saro9EdlMcArjPdnOizHZZs1HxhhjIqymYIwxJsJqCsYYYyIsKBhjjImwoGCMMSbCgoIxxpgICwrGGGMiLCgYY4yJ+P8ggP1SiZDSoQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#plot scores\n",
    "with open(\"scores_file.json\", \"r\") as read_file:\n",
    "    scores, scores_avg = json.load(read_file)\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "plt.plot(np.arange(1, len(scores)+1), scores)\n",
    "plt.plot(np.arange(1, len(scores)+1), scores_avg)\n",
    "plt.ylabel('Score')\n",
    "plt.xlabel('Episode #')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Inference\n",
    "DEVICE = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
    "seeding()\n",
    "state_size=env_info.vector_observations.shape[1]\n",
    "action_size=brain.vector_action_space_size\n",
    "num_agents=env_info.vector_observations.shape[0]\n",
    "MADDPG_obj = MADDPG(state_size=state_size, action_size=action_size, num_agents=num_agents)\n",
    "MADDPG_obj.load_maddpg() #load the local network weights\n",
    "\n",
    "scores_list = []\n",
    "def MADDPG_Inference(n_episodes=3, t_max=1000):\n",
    "    for i_episode in range(1, n_episodes+1):\n",
    "        env_info = env.reset(train_mode=False)[brain_name]     # reset the environment\n",
    "        states = env_info.vector_observations                   # get the current states (for all agents)\n",
    "        MADDPG_obj.reset() #reset the MADDPG_obj OU Noise\n",
    "        scores = np.zeros(num_agents)                          # initialize the score (for each agent in MADDPG_obj)\n",
    "        for _ in range(t_max):\n",
    "            actions = MADDPG_obj.act(states, i_episode=0, add_noise=False)\n",
    "            env_info = env.step(actions)[brain_name]           # send all actions to the environment\n",
    "            next_states = env_info.vector_observations         # get next state (for each agent in MADDPG_obj)\n",
    "            rewards = env_info.rewards                         # get rewards (for each agent in MADDPG_obj)\n",
    "            dones = env_info.local_done                        # see if episode finished\n",
    "            scores += rewards                                  # update the score (for each agent in MADDPG_obj)\n",
    "            states = next_states                               # roll over states to next time step\n",
    "            if np.any(dones):                                  # exit loop if episode finished\n",
    "                break\n",
    "        print('Episode {}: {}'.format(i_episode, scores))\n",
    "        scores_list.append(np.max(scores))\n",
    "    print('Mean score is: ', np.mean(np.array(scores_list)))\n",
    "\n",
    "MADDPG_Inference()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "env.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
